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ABSTRACT 

Warfighting Commanders in Chief (CINCs) have identified a need to provide lower- 
level tactical units (especially brigades) with real-time responsive Reconnaissance, 
Surveillance, and Target Acquisition (RSTA). There are many unanswered questions, some 
of which are: “Which UAV system best suits the needs of the brigade commander?”, “How 
many UAVs does a brigade need?”, and “What are the Tactics, Techniques, and Procedures 
(TTP) for the use of this new system?” This thesis demonstrates the ability to design a small 
high resolution simulation which can be used to answer these questions. The simulation can 


be used throughout the acquisition process, and potentially beyond. 
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THESIS DISCLAIMER 


The reader is cautioned that computer programs developed in this research may not 
have been exercised for all cases of interest. While every effort has been made, within the 
available time, to ensure that the programs are free of computational and logic errors, they 
cannot be considered validated. Any application of these programs without additional 


verification and validation is at the risk of the user. 
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EXECUTIVE SUMMARY 





In 1996, General John M. Shalikashvili, former Chairman of the Joint Chiefs of Staff, 
released Joint Vision 2010, which outlines a conceptual template for the evolution of the 
Armed Forces of the future and a pending Revolution in Military Affairs (RMA). A key 
element of this evolution and revolution focuses on the exploitation of "information-age" 
technological advances. More lethal weapon systems, new sensor packages, and improved 
information transfer are now possible because of increasingly sophisticated technologies. 
These new technologies are so powerful that they are revolutionizing the art of warfare for our 
Armed Forces. 

One of these new technologies is the unmanned aerial vehicle (UAV). The Army's 
Training and Doctrine Command (TRADOC) is examining the impact of this new technology 
through Advanced Warfighting Experiments (AWE). However, there are nies questions 
that must be answered as the Army moves toward acquisition and fielding. Simulation is an 
important decision aid in these efforts. Unfortunately, many traditional simulation models are 
not robust enough to reflect many of the attributes of this new system. In addition, these 
simulations are difficult to modify and require a significant amount of time and money to 
modify. 

This thesis demonstrates the ability to design a small high resolution simulation which 
can be used to answer questions about the performance of UAV systems that arise before, 
during, and after the acquisition process. Furthermore, this thesis establishes a basis for 
further research in the analysis of UAV performance and effectiveness. 

The simulation developed was written in Java and uses the discrete-event simulation 


library Simkit which is available from the Naval Postgraduate School. The model allows 





analysts to answer specific questions about the performance of UAVs. The structure of the 
model is based on eat graphs in which nodes represent events and their connecting arcs 
represent the passage of time. This simulation is a stochastic, event-step model. 

This work demonstrates the ability to use the model to determine the values of 
measures of performance (MOP) and the effect of modifying performance parameters. 
Threshold and objective values specified in the Operational Requirements Document (ORD) 
are examined for their adequacy for the acquisition of systems. The point of diminishing 
retum is determined as well and the benefit of structural changes. Lastly, the ability for the 
analyst to perform analysis of alternatives is demonstrated. This allows for the comparison of 


existing and future UAV systems. 
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I. INTRODUCTION 


A. GENERAL. 

The past few years have been significant in the history of the United States Armed 
Forces. In 1996, General John M. Shalikashvili, former Chairman of the Joint Chiefs of Staff, 
released Joint Vision 2010, which outlines a conceptual template for the evolution of the 
Armed Forces of the future and a pending Revolution in Military Affairs (RMA). A key 
element of this evolution and revolution focuses on the exploitation of "information-age" 
technological advances. More lethal weapon systems, new sensor packages, and improved 
information transfer are now possible because of increasingly sophisticated technologies. 
These new technologies are so powerful that they are revolutionizing the art of warfare and 
marking a "strategic inflection point” for our Armed Forces [Ref. 1: p. 4]. Throughout 
history, employment of technologically superior equipment with the appropriate Tactics, 
Techniques, and Procedures (TTPs) and correct processes has been critical to the success of 
our forces [Ref. 2:p. 7]. Hence, "we must change in order to sustain current levels of 
excellence in the future" [Ref. 3:p. 1]. 

Information technologies and their impact on future military operations are an 
important theme throughout Joint Vision 2010. Our forces must attain "information 
superiority." Joint Vision 2010 defines information superiority as "the capability to collect, 
process, and disseminate an uninterrupted flow of information while exploiting or denying an 
adversary's ability to do the same" [Ref. 2:p. 16]. By achieving information superiority, our 
forces can achieve "dominant battlespace awareness,” an interactive "picture" which yields 


accurate assessments of friendly and enemy operations within the 





area of interest [Ref. 3:p. 13]. An apparent challenge is developing a means to attain dominant 
battlespace awareness in the midst of significant force structure modifications. 

On 9 June 1998, General William Hartzog, then Commander of Training and Doctrine 
Command, put the Army's stamp of approval on the "new heavy division" design. The "new 
heavy division will still be the most lethal combat force in the world, even though it will have 
fewer soldiers and armored vehicles" [Ref. 4:p. 1]. Each maneuver brigade will lose a mixture 
of three companies’ worth of armor and infantry vehicles, a significant decrease in combat 
power. However, the Army still expects this force to cover about three times more battlefield 
area than present day divisions. Technological advances in better sensors that will facilitate 
increased situational awareness are part of the Army's answer to this potential decrease in 
lethality and force isacln 

Previous TTP did not require that all of a brigade commander's tanks and Bradley 
Fighting Vehicles (BFV) be engaged with the enemy. Most often, a commander would 
establish a screening or guard force as well as reserves because he was unsure of the enemy's 
location and his alternate avenues of approach. With better sensor packages and improved 
means of gathering intelligence, "you don't have to worry about that other direction anymore. 
Now a commander can focus all his energy in one direction’ [Ref. 4:p. 2]. One of the many 
- enabling technologies that will allow commanders to achieve better visibility of the battlefield 
and dominant battlespace awareness is the Tactical Unmanned Aerial Vehicle (TUAV). It is 
the structure and characteristics of the TUAV system that is the focus of this thesis. 

The TUAV can now deliver this "picture" to the warfighter because of advances in 


technology and TTPs. Smaller, cheaper TUAV platforms and sensor packages, previously not 


available, now give the battlefield commander the capability to achieve a near-real time 

















picture of the battlefield. This information, processed accurately and efficiently, may allow 
the commander to achieve “dominant maneuver” and “precision engagements, ” two of the 
emerging concepts outlined in Joint Vision 2010. "Dominant maneuver". is the 
multidimensional application of information, engagement, and mobility capabilities to 
position and employ widely dispersed joint air, land, sea and space forces to accomplish the 
assigned operational tasks [Ref. 2:p. 20]. "Precision engagements” consist of a system of 
systems that enables our forces to locate the objective or target, provide responsive command 
and control, generate the desired effect, assess our level of success, and retain the flexibility to 
reengage with precision when required [Ref. 3:p. 21]. The TUAV is a combat multiplier and 
is essential to achieving information superiority and dominant battlespace awareness which 
will facilitate "dominant maneuver" and "precision engagements” [Ref. 5:p. 1]. 
B. PROBLEM DESCRIPTION. | 
UAV systems currently exist and have been employed in a number of military 
spciations: However, these systems have primarily supported higher-level units and national 
agencies, such as Army Corps and the Central Intelligence Agency (CIA) [Ref. 6:p. 1]. In 
1998, the Army began the process of acquiring a UAV for brigade level units with the 
submission of the Operational Requirements Document (ORD) for the Close Range — Tactical 
Unmanned Aerial Vehicle (CR-TUAV). There are numerous potential TUAVs from which to 
choose. Immediate questions of interest are: “Which TUAV best suits the needs of the 
brigade commander?”, “How many TUAVs does a brigade need?”, “What are the Tactics, 
Techniques, and Procedures (TTP) for this new system?”. The larger questions are: “How 
effective is a TUAV system as an integrated part of sensor-shooter links? ” and "How must 


current processes be modified to achieve maximum effectiveness?" 





The answers to these questions could be obtained through extensive field testing and 
experimental training with prototypes; however, such an approach would require an 
overwhelming amount of money and time. A more practical solution is evaluation and 
analysis through simulation. In recent years there has been greater emphasis on the use of 
simulation in the acquisition process. 

Since October 1995, Dr. Paul Kaminski, the Under Secretary of Defense for 
Acquisition and Technology, has required that the Simulation, Test and Evaluation Process 
(STEP), a concept of repetitive cycles of “model, test, model,” be an integral part of the test 
and evaluation process [Ref. 7:p. iti]. This requires a decision: "which simulation should be 
used?" One could use an existing high-resolution model; however, according to the Defense 
Modeling and Simulation Office (DMSO), most present models are narrowly focused, too 
costly to operate, and not easily extensible. For example, Janus could be used for such an 
analysis; however, it does not explicitly model TUAVs. sales can use existing entities to 
perform like TUAVs, but such efforts would only be a "work around." Such limitations are 
inadequate for the purposes of this study. Modification of Janus to incorporate new 
technologies such as TUAVs and future technologies and capabilities available to TUAVs 
would require a tremendous amount of effort and money [Ref. 8:p. 30]. Such models may not 
be the best alternative for this type of analysis. The approach taken in this thesis is to use a 
smaller high-resolution model based on the component approach. This type of model does not 
require much time to develop and allows for reuse and dynamic changes that permit more in 


depth analysis. 








Cc PURPOSE. 

The goal of this thesis is to evaluate the performance of TUAVs by developing 
simulations that will support system developers and assist decision-makers in the acquisition 
process. This thesis has a dual purpose. The first purpose is to evaluate measures of 
performance (MOP) of existing or proposed TUAV systems. The second purpose is to 
establish a basis to evaluate measures of effectiveness (MOE) of a TUAV system as part of. 
sensor-shooter links within a scenario of interest. Finally, the important question to answer is: 

How effective is a TUAV system in supporting a brigade's mission? 

D. SCOPE. 

The purpose of this thesis is not necessarily to give a specific answer to the above 
question but to produce a simulation tool as a proof of concept to assist decision-makers in 
answering important structural and operational requirement questions about TUAV systems. 
By manipulating input, we can conduct parametric analysis to determine possible threshold 
values for key technical parameters. Also, by using parameters for existing and/or future 
TUAV systems, this tool can be used to perform an analysis of alternatives. 

This study uses discrete event simulation models where the entities are constructed 
using the component approach. Benefits of this methodology include the potential for 
scalability and reuse. Analytic (mathematical) models as well as other Monte Carlo 
simulations are used to assist in verifying the portion of the model that evaluates the 
performance of TUAV systems. Further considerations are examined and suggestions are 


made for continuing work on this important problem. 





E. THESIS STRUCTURE. 

This thesis consists of six chapters. This first chapter has been an introduction to the 
problem and the content of the thesis. The second chapter covers the background of the 
problem in order to give the reader a better understanding for the motivation of the thesis 
work. The third chapter focuses on the portion of the simulation that evaluates the 
performance of UAVs. Additionally, a discussion of verification is presented. Chapter four 
discusses performance of a system and analysis of alternatives. The last two chapters present 


recommendations and offer a conclusion. 




















Il. BACKGROUND 
...no longer is there any doubt that UAVs will play a major military role 
whether it be in open conflict or peacekeeping. 
-Rear Admiral Barton D. Strong 
Head of the Joint Projects Office for Cruise Missiles and Unmanned Aircraft 
A. HISTORICAL DEVELOPMENT. 
The idea of using UAVs in military operations is not new. As early as World War I 
UAVs, commonly referred to as “drones,” were used as aerial targets and for belligerent 
purposes. UAVs have been used as reconnaissance assets since the 1920’s and more recently 
during the Korean and Vietnam Wars. The Lightning Bug UAV was one of only two aircraft 
to fly reconnaissance missions in North Vietnam [Ref. 9:p. 3]. In 1979, the Army fielded the 
first major UAV acquisition, the Aquila. However, this program was canceled because of 
cost, delays, and technical difficulties [Ref. 10]. Military spertions in Grenada, Lebanon, and 
Libya highlighted the need for an on-call, inexpensive reconnaissance and Battle Damage 
Assessment (BDA) capability for local commanders. Consequently, the Secretary of the 
Navy directed acquisition of UAVs for the Navy in July 1985. The Army acquired and 
fielded the Pioneer system in 1990. Since Pioneer's debut, it has been used in military 
operations ranging from the Gulf War to Peace Keeping Operations in Bosnia [Ref. 11:p. 3]. 
Historically, interest in UAVs has risen and fallen. In the past few years, interest has 
continuously increased for two reasons. The first is the heightened sensitivity to risking 
human life in combat. During the Cold War, the U.S. flew reconnaissance missions over the 
Soviet Union. In May 1960, Francis Gary Powers' U-2 spy plane was shot down [Ref. 12]. 


During the Cuban Missile Crisis in October 1962, Rudolph Anderson's U-2 was shot down 





and crashed in the Cuban jungle [Ref. 13]. These incidents sparked national interest in the use 
of alternative, unmanned means of gathering intelligence. The Air Force and other national 
agencies then directed resources into UAV programs [Ref. 14:p. 34]. The UAV was 
identified as a relatively cheap alternative "when measured against the politically risky 
alternatives of a soldier's death or capture while conducting intelligence operations" [Ref. 5:p. 
1}. 

The second major reason for increasing interest in UAVs is the information-age 
technological advances that are the foundation of Joint Vision 2010 as discussed in Chapter I. 
It has been said, "An unadulterated picture still tells a thousand words." The UAV can now 
deliver this "picture" to the warfighter because of advances in technology along with TTP. 
Smaller, cheaper UAV satis and sensor packages now allow the battlefield commander 
the capability of achieving a near-real time picture of the battlefield. This information, 
processed accurately and efficiently, may allow the commander to achieve “dominant 
maneuver" and "precision engagements." | 

To attain dominant maneuver and precision engagement, information superiority must 
be achieved [Ref. 15]. The UAV has played a major role in acquiring information superiority 
in such operations as the Persian Gulf War, Task Force XXI’s deployment to the National 
- Training Center (NTC), A" Infantry Division Advanced WarFighting Exercises (DAWE), and 
operations in Somalia and Bosnia. At the conclusion of the Gulf War, Lieutenant General 
Boomer, USMC Central Command, praised the Pioneer system as "the single most valuable 
intelligence collector" [Ref. 11:p. 3]. During an Advanced Warfighting Experiment (AWE) at 
the NTC, blue forces were equipped with TUAVs while the red forces were not. At the 


conclusion of the AWE, the opposing force commander was asked, "if you could take one 




















system away from the blue forces, what would it be? The answer was the TUAV" [Ref. 5:p. 
2). 

The tactical UAV is absolutely critical to our brigade and division 

commanders.... It is their confirming sensor, and the "eyes" which enable 

commanders to see critical portions of their battlefield and target anything 

they can see. 

-Lieutenant General Paul E. Menoher, Jr. 
Deputy Chief of Staff for Intelligence, U.S. Army 
5 August 1996 
UAVs will most certainly play a major role in future military operations. 

The UAV systems employed previously in "real world" operations have supported 
higher-level units, i.e. - Corps. With the signing of the Mission Need Statement (MNS) for 
Close Range Reconnaissance, Surveillance and Target Acquisition (RSTA), warfighting 
Commanders in Chief (CINCs) have identified a need to provide lower-level tactical units 
with real-time responsive RSTA. On 25 February 1999, the Military Intelligence Center UAV 
Operations Office submitted the ORD for the “Brigade Commander’s UAV.” This document 
outlines MOPs for a UAV that will fulfill the MNS. Still, there are many unanswered 
questions, some of which are: “Which UAV system best suits the needs of the brigade 
commander?”, “How many UAVs does a brigade need?”, “What are the TTPs for the use of 
this new system?” In addition, MOEs will be identified to answer: "How effective is a 
system?" Many answers to these questions can be obtained through models developed as part 


of Simulation Based Acquisition (SBA) using STEP. 





B. SBA AND STEP. 

SBA is efficient integration of modeling and simulation tools and technology in the 
acquisition process. The goals of SBA are [Ref. 7:p. 1-2]: . 

e Substantially reduce time, resources, and risk associated with the acquisition 

process 

e Increase the quality, military utility, and supportability of fielded systems while 

reducing total ownership costs 

e Enable Integrated Product Process a (IPPD) across the full acquisition 

life cycle. 

STEP is the integration of modeling and simulation with test and evaluation. More 
specifically, STEP is an interactive process of "model-simulate-fix-test-iterate," with the 
results of tests feeding back into the model [Ref. 7:p. 6]. This necessitates the existence of 
models that can be easily modified; moreover, such models should be dynamic and reusable. 
A program manager should be able to use the same model by making easy modifications 
throughout the acquisition process. He should not have to start over from scratch to answer 
additional questions [Ref. 16:p. 38]. 

In an effort to find a model which could be used to evaluate MOPs for a UAV system, 
two simulations were discovered, both developed by the Institute of Defense Analyses (IDA). 
The first model, a discrete event simulation using Extend™ was developed in support of the 
Director, Operational Test and Evaluation (DOT&E). It was developed to assist in the 
analysis of the effective time on station (ETOS) for the Predator UAV [Ref. 17]. A second 
model, the Military Aircraft Sustainability Simulation (MASS) is an expansion of the 


Extend™ model and can provide analysis for a variety of platforms [Ref. 18]. 
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There are two limitations of MASS that must be addressed to make it more robust and 
therefore better suited for analysis of the TUAV. Although MASS is coded in C++ and is 
object-oriented, it is not component-based. Hence it is difficult to incorporate new features 
_ and/or modify entities that would be essential for an analysis of a TUAV. Secondly, several 
assumptions are made in MASS that are unacceptable for performance analysis in this thesis. 
Examples are no crew-related limitations and perfect maintenance. 

Developing a simulation using Java and Simkit provides a more flexible component- 
based srvuiladon that will allow easier modification of features and capabilities. A brief 
explanation of Simkit and the code can _ be _ obtained at URL 
http://diana.or.nps.navy.mil/~ahbuss/OA3302W99/. Additionally, since the model was 
created in Java it can be executed on virtually any platform and run from the Internet. 
UAVSim is a simulation that extends the MASS simulation to allow for more robust 
performance evaluation of TUAV systems. UAVSim models Ground Control Station (GCS) 
and maintenance crew related limitations, maintenance prioritization, and operational tempo 
(OPTEMPO) requirements. It allows for non-perfect maintenance and the ability for the user 
to select distributions. The aforementioned were identified as "future enhancements" to the 
MASS model [Ref. 18]. With such an improved model, decision-makers will es better 


informed in the UAV acquisition process. 
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I. MODEL DEVELOPMENT 





A. MODEL DESCRIPTION. 

The performance evaluation portion of UAVSim is a stochastic, discrete event 
simulation written in Java using Simkit, a discrete event simulation package. This simulation 
is an abstraction of the events a TUAV system encounters during a deployment. The model 
allows for the evaluation of a system and alternate system(s) by modifying input parameters. 
A stochastic simulation model, such as UAVSim, introduces uncertainly or randomness by 
drawing a random observation from a distribution specified by the analyst [Ref. 19:p. 32]. 
Randomness was introduced into the model because the "real world" system that UAVSim 
models also involves randomness. For example, the time between mission affecting failures, 
ground repair and non-mission affecting failures vary from flight to flight of the UAVs and 
can only be simulated using the appropriate probability distribution and estimates of that 
distribution's parameters. Each of these times are an integral part of the system and the effect 
of their randomness on the "real world" system can best be approximated by introducing 
randomness in the model. The measures of performance (MOP) which this thesis attempts to 
estimate are accordingly also random. 

For the purposes of this thesis, any TUAV sii is considered to be part of a UAV 
‘company. This modeling decision reflects the way these systems actually support a brigade or 
division. For example, in the 4" Infantry Division the TUAVs supporting the brigades and 
division are part of a company within the 15" Aerial Exploration Battalion. We assume the 
company consists of the same type of TUAV. Within the company, there is a baseline of 


UAVs, Ground Control Station (GCS) and maintenance section as shown in Figure 1. 
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UAV Company 


Baseline Maintenance Ground Control Station 
Section (GCS) 
UAV 
(1 ton) I 


Figure 1: UAV Company Structure 


1. BaseLine. 

The term baseline refers to the collection of UAVs that are within the UAV company. 
In this study the platforms (the term platform or vehicle is a general term that refers to the 
TUAV) that compose the baseline are all of the same type. Since attrition is not modeled, the 
number of UAVs in the baseline remains constant; however, adding attrition which causes the 
number of UAVs in the baseline to change can easily enhance the model. 

2. GCS. 

The brain of the company is the GCS and controls the flight of all platforms. Within 
the model, it is responsible for scheduling, limiting the number of UAVs that can be flying at 
any given time, maintaining an "audit trail" of the actions of the company, and collecting 


statistics on MOPs. 














3. Maintenance Team. 

The company's maintenance section is responsible for servicing the platforms and 
sensors. This model allows for the existence of single or multiple maintenance paths that 
perform the same types of maintenance, routine or scheduled. Each service is based on a first- 
in, first-out (FIFO) queue. Also, once a platform has been serviced, it is considered "as good 
as new." It is assumed the UAVs are not permitted to exhibit "wear and tear." 

B. MODEL EVENTS AND DESCRIPTION. 
UAVSim models the different events a TUAV encounters during a deployment. 


Figure 2 shows these processes. 





Figure 2: Event Graph of UAVSim. 
A detailed discussion of event graphs and their use in simulation modeling is 
presented in "Modeling with Event Graphs" [Ref. 20]. The reader should not 
consider an event graph the same as a flow diagram. 
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A UAV starts by being assigned a mission, removed from the company's baseline, and 
launched. The UAV ingresses for a pre-planned amount of time in order to reach a specified 
region, the area of operation (AO). Upon arriving in the AO, the UAV observes the area for a 
pre-planned amount of time and then egresses. In this simulation, only one UAV is permitted 
to search or operate in the AO. The amount of time a UAV egresses is the same as that spent 
ingressing. Once the UAV lands, it proceeds to the maintenance section where the type of 
maintenance required is determined. If routine maintenance is to be performed, the 
maintenance time is exponentially distributed plus a fixed amount of time for logistics delay. 
If a scheduled maintenance is to be performed, the maintenance time is a pre-determined value 
plus the logistics delay. Only one vehicle can be serviced at a time on a given path. 

During flight, the platform is susceptible to mission affecting failures (MAF) and non- 
MABFs. If a MAF occurs while the platform is ingressing or performing its mission, the UAV 
immediately egresses and proceeds to the maintenance section for servicing provided a 
maintenance path is available. If a path is not available, the UAV queues for maintenance. It 
is assumed that the distribution for all MAF repairs is independent and identically distributed 
(iid) exponential. The random times from take-off to the occurrence of a MAF are also 
considered to be iid exponential; thus UAVs always — a mission "as good as new." 

Non-MAFs are the second type of failures that can occur while a UAV is flying; 
however, these failures do not cause the vehicle to egress. The occurrence of this type of 
failure is modeled as the counts of a Poisson Process. UAVSim "listens" to the component, 
PoissonProcess, for the occurrence of a non-MAF and increments the number of non-MAFs 
which have occurred. Figure 3, shows this process. The accumulation of this _ of failure 


only increases the required maintenance time. 
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PoissonProcess Component 


UAVSim 





Figure 3: Occurrence of Non-MAFs Model 





Similar to MAFs, the repair distribution for non-MAFs is assumed to be iid 
| | exponential. To eliminate this assumption, one could use data from operational tests and. 
determine the distributions that are required. Due to time constraints and lack of data, this 
technique was not used. 

After maintenance, the UAV enters the turn phase where it spends a pre-determined 


amount of time called "Turn Time" (TT). The purpose of the turn phase is to administratively 





prepare the platform for the next mission. Following completion of this phase, the UAV 
enters the baseline and awaits orders to perform its next mission. 


Brief descriptions of the events shown in Figure 2, are presented in Table 1. 
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This is the first event that a UAV executes. A UAV is taken 
out of the baseline and immediately begins ingressing. 

The UAV flies toward the AO to perform surveillance. While 
the UAV is ingressing, it may have a MAF. If so, the UAV 
immediately aborts and egresses. 
Once a UAV begins ingressing, the launch of the next UAV is 
scheduled such that as the current UAV begins egressing, the 
next UAV begins performing its mission. If the current UAV 
aborts the mission, a UAV is launched immediately given that 
one is available. However, if a UAV is not available, the AO is 
not covered until the next available UAV can arrive on station. 
When a UAV has a MAF during ingress, it aborts and 
immediately egresses. Since the platform did not travel for the 
full ingress time, the egress time is adjusted accordingly. 
The UAV successfully begins coverage of the assigned region; 
however, it may have a MAF forcing a Mission Incomplete 


LaunchNextUAV 


MissionAborted 
. | Status. 
If a UAV has a MAF while it is providing surveillance of the 


AO, it immediately egresses. 
- | that occurs has no effect on the flight of the UAV. 


Land The UAV stops flying, enters the maintenance queue and is 
prepared to enter maintenance. If another UAV should have |. 
| been launched earlier but could not because of limitations on 
. the number of UAVs flying, a UAV is due to be launched. 
| Provided there are UAVs available to be launched, a signal 1s 
sent to the "Launch" method. 


Either the UAV enters routine or scheduled maintenance. If the 
platform has flown less than a designated amount of time, the 
platform enters routine maintenance. If the UAV has flown 
more than a designated amount of time, the platform enters 
scheduled maintenance. Once a platform begins this event, the 


StartMaintenance 
maintenance queue. In this version of the model, a UAV 
cannot skip maintenance. 


number of available maintenance paths is decreased. If no 
maintenance paths are available, the UAV enters the 
The maintenance section performs routine maintenance on the 
UAV. 
StartScheduledMaintenance | The maintenance section performs scheduled maintenance on 
| the UAV. | 7 
EndMaintenance The UAV exits the maintenance queue. If there are UAVs 


| awaiting maintenance, a signal is sent to "StartMaintenance." 






































































The platform has completed maintenance and is 
administratively prepared for the next mission. The amount of 
time spent in this phase is deterministic. 
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The platform is prepared for the next mission, enters the 
company baseline and awaits instructions to begin the next 
mission. If another UAV is due to be launched, a signal is sent 
to the "Launch" method. 
The arrival of non-MAF is modeled as a Poisson Process. 
UAVSim is linked to another component, PoissonProcess that 
is running simultaneously. UAVSim "listens" for the arrival of 
non-MAFs. The number of non-MAFs that occur while a 
UAV is flying is recorded and used to adjust the amount of 
time that UAVs spend in maintenance. 


















Table 1: Description of Events in Version 1.0 of UAVSim | 


C. | MODEL ASSUMPTIONS. 

In the development of this model, several initial assumptions were made to decrease 
the complexity of the model and allow comparison between UAVSim and MASS. If the 
models compared are run using similar assumptions and input parameters, each model should 
return similar results. It should be noted that some of these assumptions will be relaxed when 
the model is expanded to handle more complex issues appropriate for modeling TUAVs. The 


initial assumptions that were made are presented in Table 2. 
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1. No Attrition or Loss of Platforms No attrition or loss of platforms is modeled. 
Only MAF cause a platform not to perform 
its mission. 


2. No Weather-Related Effects Bad weather (heavy rain, snow) does not 
affect the performance of the UAV. 


3. No Ground aborts No ground aborts occur. Once a UAV is 
dedicated to ingress, it will ingress. No 
MAFs occur before launch. 


4. Distributions of MAF and Repair Times | All classifications of MAF (human, 
mechanical, etc) are aggregated into the 
general classification of MAFs. The times. 
between MAFs are modeled as iid 
observations from an exponential distribution. 
Additionally, repair times are modeled as tid 
observations from an exponential distribution. 
These distributions remain constant 
throughout the deployment. 
At the conclusion of maintenance actions, 
UAVs are 100% mission capable; platforms 
receive perfect maintenance and. do not 
exhibit "wear and tear." 


6. No Function Checks After completion of the maintenance phase, 
no function checks are performed. 


7. No Crew-Related Limitations Crew-related limitations such as numbers of 
pilots/payload operators, flight hours that 
pilots/payload operator can fly, numbers of 
‘| mechanics or the number of hours that 
mechanics can work are limiting factors. 
None of these limitations are modeled 
initially. 






































5. No Wear and Tear 

















Table 2: Initial Model Assumptions 


D. MODEL INPUTS. 
The input parameters for UAVSim are very similar to those used in MASS. This 


similarity further facilitates the development of a model like MASS. The initial inputs for the 


model and a brief description are given in Table 3. 
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Number of UAVs The number of UAVs in a 
baseline... 


Number of Maintenance Paths The number of paths 
available to service 
platforms. Maintenance 
paths are equivalent to 
servers. 


Maximum Number of UAVs in Flight The GCS can only control a 
| specified number of UAVs 
at a given time. 


Ingress Time (hours) The length of flight from the 
launch and recovery site to 
the region to be observed. 

The amount of time the 
platform requires to reach 
the launch and recovery site 
from the position where it 
begins egressing. 


Scheduled Time on Station (hours) The pre-planned amount of 
time a platform is scheduled 
to spend in the AO. 


Number of Deployments The number of replications 
of the simulation. One run 
may consist of several 
replications or deployments. 














Egress Time (hours) 














Platform Turn Time (hours) The time required to prepare 
a UAV for a_ subsequent 
launch. TT is a constant. 


Logistics Delay Time (hours) 





The length of time required 
to obtain parts, for a piece of 
equipment to become 
available, etc. This time is 
added to whatever time is 
required for maintenance. 
This is a constant. 


Mean Time for Ground Repair (hours) The expected time for the 
maintenance section’—_‘to 
service a platform. 


Time to Complete Scheduled Maintenance (hours) The time associated with the 
maintenance section 
completing a scheduled 
maintenance for a UAV. 
This time is assumed to be 
constant. 
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The time required by the 
maintenance section to repair 
each non-MAF. This value is 
assumed to be a constant. 
Flight Time Between Scheduled Maintenance Actions (hours) | The amount of time the 
platform may fly before a 
scheduled maintenance. 
This time is assumed to be 
constant. 
The expected time between 
occurrences of MAFs. In 
this version of the model, the 
time between platform | 
MAFs is exponential. 


Length of Deployment (days) The length of a deployment 
or replication of the model. 


Mean Time Between Non-MAF (hours) The expected time between 
occurrences of non-MAFs. 
In this version of the model, 
the time between platform 
MAFs is exponential. 
This is the z-value from the 
standard normal distribution. 
The number specified is used 
to determine the confidence 
interval for MOPs. 


Time to Repair Each Non-MAF (hours) 






















Mean Time Between Platform MAF (hours) 


















Table 3: Description of Model Inputs 


E. MODEL OUTPUTS. 

The initial UAVSim model returns MOPs identical to those returned by the MASS 
model. This allows easy comparison of results and also facilitates comparison of the models 
to determine if UAVSim is operating as it should. The initial MOPs generated by UAVSim 


are presented in Table 4. 


Z2 








Effective Time On Station (ETOS) The mean percentage of time that the AO 
is covered by at least one UAV. 


Time Non-Mission Capable (NMC) This is the mean time platforms are non- 
mission capable given that a MAF occurs. 
This time is measured from the moment a 
MAF occurs until it has completed 
maintenance. 


Sortie Generation Rate (sorties per tme period) | The average number of launches during a 
deployment. 


Mean Wait Time in Maintenance Queue The average amount of time that UAVs 
spend waiting for maintenance given that |. 
there is more than one UAV in the 










baseline. 


Table 4: Description of Model Outputs 


Both MASS and UAVSim compute confidence intervals for each of the MOPs. As 
part of the input parameters, the user specifies the number of deployments, the sample size, 
and a confidence level by entering the appropriate value for a standard normal random 
variable, referred to as the "z-value." Each of the MOPs is the mean from several observations 
or in this case, deployments. By using the Central Limit Theorem (CLT), if the number of 
replications (deployments) of the simulation is sufficiently large, the MOP has approximately 
a normal distribution [Ref. 2p. 232]. Using point estimators for the mean and standard 
deviations, UAVSim calculates and presents a confidence interval corresponding to the "z- 
value” the user specified. 

F. VERIFICATION OF MOP NORMALITY ASSUMPTION. 

As mentioned previously, the values for the MOPs and the Bonferroni intervals were 
computed using the assumption that each observation of the MOP was iid normal. In order to 
verify this assumption, diagnostic plots and goodness of fit tests were used. The Chi-Square 
Test for Goodness of Fit and the Kolmogorov-Smirnov Test in the statistical analysis package 


S-Plus 4.5 were used. 
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1. Diagnostic Plot. 
One run of UAVSim in which the number of deployments was set to thirty was 


- completed. The ETOS for each deployment was captured and imported into S-Plus. A 
"qqnorm” plot of the data is shown in Figure 4. Each point in the figure represents the average 


of the count of total hours UAVs provided coverage divided by the total count of hours that 


coverage should have been provided. 


The plot appears to indicate that the data is close to normal; however, more robust 
analysis and determination can be completed with the goodness of fit tests. The first test 


applied was the Kolmogorov-Smimov composite test. 


~ Quantile Quantile Plot of ETOS 


probability 
0.340 


0.335 


0.330 





Quantiles of Standard Normal 


Figure 4: Quantile-Quantile Plot of ETOS 
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2. Goodness of Fit Tests. 


The composite tests were used because it is hypothesized that the data comes from a 
normal distribution but the parameters, up and o, are estimated and not known. The null and 
alternate hypotheses are: 

H,: The true cdf equals the normal distribution for all sample points. 

H,: The true cdf does not equal the normal distribution for at least one sample point. 
The values for 2 and o were estimated and an exploratory plot of the empirical and 
hypothesized distribution was pondecied: Figure B-1 at APPENDIX B. The plot indicates 
that the distributions appear to be the same. Next, applying the Kolmogorov-Smimov test 
resulted in a p-value of 0.72. Since a value of 0.05 was used for a, the null hypothesis was not 
rejected. Similarly, the p-value from the Chi-square Goodness of Fit Test was 0.68 and the 
null hypothesis could not be rejected. Given the test results and efforts, it 1s assumed that the 
observations of the MOP are tid normal. 

This will allow us to safely use confidence intervals and other inference techniques 
based on the normal distribution. 

3. Independent Observations. 

The value of ETOS was computed for each run of the simulation with tid observations 
from distributions within the model. Thus the resulting value of ETOS conditioned on the 
parameter values for a particular run is independent of the other runs. 

G. VERIFICATION OF THE MODEL. 

In the development of a model, important questions are: "Does this model do what it is 

supposed to do?” or "Has this model been verified?" Verification is "the process of 


determining that a model implementation accurately represents the developer's conceptual 
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description and specifications” [Ref. 22:p. I-3]. Verification of UAVSim was conducted using 
two independent methods. The first used an analytical model developed by Gaver, Jacobs and 
Stoneman [Ref. 22:p. 3]. The second method used a similar simulation, the MASS model. 
Two cases were explored in the verification process: single platform/single maintenance path 
(Case I) and multiple platforms/single maintenance path (Case II). It should be ead that both 
models were used to verify UAVSim in Case I; however, only the MASS model was used in 
Case II. The analytical model could not be used in Case II because of the complex scheduling 
requirement when more than one UAV is available. For each case all models use the same 
assumptions and processes. 


1. Case I Comparison. 


This analytical model assumes that the time to a MAF is exponentially distributed 
with rate 2. Upon returning to the launch and recovery site, the platform is serviced by the 
maintenance section where the time to repair is exponentially distributed with rate p. The 
formulas to find the long run proportion of time on station (ETOS) for the analytical model are 


[Ref. 23:p. 3]: 


ao a = or ] 
; 4A Equation 1 


—[1— 6 4) + “1 ~e "40" “tl —e*]+ ED] 
L 


where T= Ingress/Egress Time 
S = On Station Time 
E[D] = Expected additional time the UAV is not flying 
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1 ] 
E[D] = a, E(B] — + @,E[B]— +— uation 2 
"By BBO 


where 1/ay = mean time between non-mission affecting failures 
E[B] = expected time from launch until landing 
1/By = time to complete each non-mission affecting failure 
1/as = time to complete scheduled maintenance actions 
1/Bs = time between scheduled maintenance actions 
1/B4 = logistics delay time + turn time 


E[B]= Ail ae ee —k -e*] | Equation 3 
where i, T, and S are as defined above 


The next step in the comparison was to enter identical parameters into the UAVSim, 


MASS and analytical models. The input parameters for the three models were those listed in 


Table 5. Also, Table 6 shows a mapping of UAVSim for the analytical model. 


Number of platforms: 1 

Number of maintenance paths: 1 

Maximum number of platforms in flight :2 

Length of deployment (hours): 2160.0 

Number of simulated deployments: 50 

Ingress time (hours): 1.0 

Egress time (hours): 1.0 

Scheduled on station time (hours): 18.0 

Platform turn time (hours): 0.0 

Logistics delay (hours): 0.5 

Mean time for ground repairs (hours): 1.9 

Time to complete scheduled maintenance (hours): 7.0 

Time to complete each non-mission affecting failure (hours): 1.9 
Flight time between scheduled maintenance actions (hours): 50.0 
Mean time between mission affecting failures (hours): 25.0 

Mean time between non-mission affecting failures (hours): 5.0 
Endurance is limited to 20 hours (ingress + egress + tSOS < endurance) 





Table 5: Model Inputs for Case I 
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T = 1.0 (ingress/egress) 
S = 18.0 (scheduled time on station) 


1/X. = 25 (mean time between mission affecting failures) 

1/u = 1/1.9 (mean time for ground repairs) 

1/ay = 5.0 (mean time between non-mission affecting failures) 
1/By = 1.9 (time to complete each non-mission affecting failure) 
1/as = 7.0 (time to complete scheduled maintenance actions) 
1/Bs = 50.0 (time between scheduled maintenance actions) 

1/Ba = 0.5 (logistics delay time + turn time) 





Table 6: Example Mapping of UAVSim Inputs. 
This mapping is for the very first calculation in Table 7. 

The parameters in bold in Table 5 were modified for each run. T was the 
ingress/egress time and tSOS was the scheduled time on station. The sum of the 
ingress/egress times and scheduled time on station was limited to 20 hours. For example, 
when T = 1, the ingress and egress time is set to one hour. Since the TUAV only has eas 
total hours endurance, there are only eighteen remaining hours for on station time. 

The results of the single platform comparison are shown in Table 7 and Figure 2: 
Note that for each run, a different ingress/egress and scheduled time on station was used. 
Also, each run encompassed fifty deployments, hence the sample size was fifty. The length of 
each deployment was 2,150 hours which is the same length used for analysis performed with | 


MASS. 
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tSOS| Analytic MASS ETOS UAVSim ETOS 
ETOS | (95% Bonferroni Interval) 

















2. 
0.2576 0.2502 - 0.2691 0.2456 - 0.2790 
| 8.0 | 0.2027 0.2008 - 0.2164 0.1937 - 0.2199 


Table 7: Case I Comparison Data 


\wow 





Comparison of Models 





Ingress Time (hrs) 





|i Analytical HMASS QUAVSim 


Figure 5: Case I Comparison of Models 


The values for ETOS from the analytical model and MASS and UAVSim simulations 
appear to agree well. Specifically, the values from the analytical model are within the 
Bonferroni confidence intervals generated by UAVSim. Also, each of the confidence 
‘itera from the two simulations overlap for all 6 cases which indicates that the simulations 


do not have results that are significantly different. 
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2. Case If Comparison. 

A comparison of UAVSim and MASS was conducted for the case of multiple UAVs 
and a single maintenance path. The inputs for both models are listed below. Note that the 
number of UAVs was set at four and the number of maintenance paths was limited to one. In 
this situation, there was the possibility for congestion or wait time when UAVs must be 


serviced. The input data for this comparison is shown in Table 8. 


Number of platforms: 4 

Number of maintenance paths: 1 

Maximum number of platforms in flight :2 

Length of deployment (hours): 2160.0 

Number of simulated deployments: 50 

Ingress time (hours): 1.0 

Egress time (hours): 1.0 

Scheduled on station time (hours): 18.0 

Platform turn time (hours): 0.0 

Logistics delay (hours): 0.5 

Mean time for ground repairs (hours): 1.9 

Time to complete scheduled maintenance (hours): 7.0 

Time to complete each non-mission affecting failure (hours) : 1.9 
Flight time between scheduled maintenance actions (hours): 50,000.0 
Mean time between mission affecting failures (hours): 25.0 

Mean time between non-mission affecting failures (hours): 50,000.0 
Endurance is limited to 20 hours (ingress + egress + tSOS < endurance) 





Table 8: Model Inputs for Case II 


The results of the MASS and UAVSim simulations are presented in Table 9. Note 
that for each run, a different ieee and scheduled time on station was used. Also, 
each run consisted of fifty deployments, as a result the sample size was fifty. The length of 
each deployment was 2,150 hours which. is the same length used for analysis purposes with 


MASS. 
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T | tSOS MASS ETOS UAVSim ETOS 
(hrs) | (hrs) |(95% Bonferroni Interval)| (95% Bonferroni Interval) 


_ 5 {| 100 | 06489-06887 | 0.5932 -0.6634 
6 | 80 | 05184-05539 | 04844-05418 


Table 9: Case I! Comparison Data 







The Bonferroni confidence intervals from both the MASS and UAVSim models are 
relatively close and overlap in all but the first case. As in Case I, it appears that there is no 
significant difference in the models. The simulations are yielding similar results. 

Thus far, it has been shown by comparison to two independent methods, the analytical 
model and the MASS simulation, that there does not appear to be a significant difference in 
the resulting MOPs in comparison with UAVSim. Every effort has been made to establish the 
Same assumptions, enter identical input parameters, extract the same MOPs, and test for 
statistical differences in the MOPs. Having performed these tasks, and given their results, I 
conclude that UAVSim is since as it should. It is providing data close enough to that of 
these previously existing models so that further study can be performed. 

H. EXPANSION OF THE MODEL. 

As discussed in Chapter IL, the present version of MASS must be expanded for 
evaluation of tactical UAVs. Specifically, sensor package failures, enhanced maintenance 
system, maintenance prioritization, non-perfect maintenance, GCS and maintenance crew- 
related limitations, ability to specify distributions and the ability to perform non-continuous 
operations must be added. A listing of each of the features that were added to UAVSim is 


presented in Table 10. 


3] 









The user can specify the distribution and parameters for the 
sensor package failure time. All sensor package failures are 
treated as MAFs and as such cause the UAV to 
immediately egress. 
The maintenance portion of the model was expanded to 
better ascertain the type of service required and appropriate 
maintenance repair time. 
The user has the option to select priority maintenance. If 
priority maintenance is chosen, maintenance is performed 
based on the required maintenance time. Lower required 
maintenance times have priority. | 


Non-Perfect Maintenance When UAVS are serviced, they are not "as good as new." 


GCS Crew Limitations The number of pilot/payload operator teams that are 
available can limit UAV operation. Flight hour constraints 
are listed in U.S. Army Regulation 95-XX [Ref. 24:p. 23]. 
The number of maintenance personnel available for service 
can limit the model. Also, the maximum number of hours 
that a team can work per day can restrict performance. 
The operator can specify the distribution and the 
corresponding parameters for: 

e Time to Platform MAF 

Time to Sensor MAF 

Repair Time for Platform MAF 

Repair Time for Sensor MAF 

Logistics Delay Time 
The user can select whether or not OPTEMPO and Surge 
OPTEMPO constraints are active. OPTEMPO 
requirements are requirements for the UAV Company to 
achieve a specified number of coverage hours per day. 
Surge OPTEMPO requirements are requirements to 
achieve a specified number of coverage hours per day but 
only for a specified number of days after which the required 
number of coverage hours for one day is less. 


Sensor Package Failures 




















Enhanced Maintenance System 











Maintenance Prioritization 






















Maintenance Crew Limitations 


















Specification of Distributioris 
















Non-Continuous Operations 


Table 10: Listing of Expanded Model Features 


I. EXPANDED MODEL EVENTS AND DESCRIPTION. 
The expansion of UAVSim was accomplished by adding additional methods to the 
existing simulation and by developing other components that UAVSim could communicate 


with while running. The addition of sensor package failures, maintenance prioritization, non- 


a2. 











perfect maintenance and specification of distributions was accomplished by adding methods to 
the existing model. However, GCS and maintenance crew-related limitations, and non- 
continuous operations were added through linking the existing model to new components. 


Figure 6 shows the structure of the expanded model. 


PoissonProcess 







MaintenanceChief 
OptempoManager | —<———______]_ SurgeManger 


Figure 6: Structure of Expanded Model 





GCSChief 





A description of each of the additions to the model follows: 

1. Maintenance Crew-Related Limitations. 

Maintenance crew-related limitations were modeled using the MaintenanceChief 
component. An event graph of this model is shown in Figure 7. The input for this component 
is the maximum number of hours that the maintenance team can work in a given day. When 
the user specifies that maintenance constraints are to be active, UAVSim and the 


MaintenanceChief communicate when a UAV requires maintenance. UAVSim sends the 
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UAV requiring maintenance to the MaintenanceChief that compares the required maintenance 
time to the available maintenance time (time remaining that the mechanic team can work). If 
the available time is greater than or equal to the required time, the UAV enters the 
maintenance path and is serviced. However, if the available time is less than the required 
maintenance time, the UAV enters the maintenance queue. If there are other UAVs in the 
maintenance queue, they are checked in the order in which they would be serviced and 
maintenance is performed if time is available. All or none of maintenance is performed; 
partial maintenance is not allowed. The amount of available maintenance time is only 
decreased when a UAV is being serviced. Otherwise, the maintenance bean is idle and 
waiting for a UAV to service. The maintenance team is on call 24 hours a day and can work 
intermittently, but less than or equal to a fixed number of hours. At the conclusion of the day, 
the available maintenance time is reset; remaining time cannot be carried over to the next day. 


The model allows only the existence of one maintenance team. 


aN 


(k) 


Maintenance 
Availability 


Not Enough 
Maintenance Time 
(k) 





Figure 7: MaintenanceChief Event Graph 
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2. GCS Crew-Related Limitations. 

GCS crew-related limitations were modeled using the GCSChief component. An 
event graph of this model is shown in Figure 8. The input for this component ‘ the number of 
GCS teams (pilot and payload operator) and the policy that governs the maximum number of 
hours per day that a GCS team can control UAVs. The policy which governs the number of 
hours per day that a GCS team can fly is outlined in AR 95-XX [Ref. 24:p. 23]. 

If the user specifies that GCS constraints are to be active, UAVSim and GCSChief 
communicate when a UAV is required to launch. UAVSim sends the UAV requiring launch 
to the GCSChief which compares the required flight time to the available flight time (time 


remaining that the GCS teams can fly). If the available time is greater than or equal to the 


(lex 


CountDay 
Ccmamay ) StartControlling 
(k) 


GCS 


Availability 


(k) 


GCSNotAvailable 


{k) 





Figure 8: GCSChief Event Graph 


required time, the UAV is launched. However, if the available flight time is less than the 
required time, the UAV remains in the baseline. If there are other UAVs in the baseline, they 


are checked and launched if crew time is available. The team is on call 24 hours a day and can 
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work intermittently, but less than or equal to a fixed number of hours. The available flying 
time is only decreased when a UAV is flying; otherwise, the crew is idle waiting to fly a 
UAV. At the conclusion of the day, the available flight time is reset according to the flight 
hour policy that is modified based on the number of days that the UAV company has been 
operating. Available time cannot be carried over to the next day. The flight policy, which was 
used in this study, is shown in Table 11. It should be noted that UAVs, which fly during the 
same time or portion of the same time, share GCS flying time. The current version of the 


model only allows the existence of one GCS team. 


14 brs 


Table 11: Flight Hour Policy 










3. Non-Continuous Operations. 

There are two types of non-continuous coverage operations and associated 
requirements: OPTEMPO requirements and surge requirements. OPTEMPO requirements 
are the number of hours that a commander desires to have coverage of an AO everyday. 
Surge requirements are a higher number of coverage hours that a commander requires for an 
AO. At the end of a specified period, the increased number of required coverage hours is 
decreased. For example, an OPTEMPO requirement may be that a commander wants to have 
twelve hours of continuous coverage every day. On the other hand, a surge requirement may 


be the commander's desire to have eighteen hours of coverage per day for three days and then 
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eight hours of coverage on the following day. The intent of the limited number of hours of 
operation on the following day is to allow the UAV company to recover. 

The OptempoManager and SurgeManager components manage non-continuous 
coverage requirements. Event graphs that show these models are in Figure 9 and Figure 10. 
When the user specifies that OPTEMPO constraints are active, UAVSim and 
OptempoManager communicate. However, when the user specifies that surge constraints are 
to be active, UAVSim, OptempoManager and SurgeManager interact. UAVSim and both 
components communicate when aUAV aegis its mission. 

The OptempoManager uses the information contained in the GCS to determine 
whether OPTEMPO constraints or OPTEMPO constraints as well as surge constraints are 
active. When only OPTEMPO constraints are active, the OptempoManager determines 
whether or not the required number of coverage hours has been achieved. If so, no additional 
UAVs are launched. Once the required coverage time has been achieved, the 
OptempoManager alerts UAVSim to egress all UAVs. When all UAVs have landed and 
completed maintenance, UAVSim determines the length of the company's down period. The 
down period is the emaing number of hours in the current day the company will be inactive. 


At the beginning of the next day, the company begins operations once again. 
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DayCounter 


EgressAHUAVs 


EndSurgePeriod 





Figure 9: OptempoManager Event Graph 


When both OPTEMPO and surge constraints are active, the OptempoManager uses 
the GCS's information as well as information from the SurgeManager. Until SurgeManager 
notifies OptempoManager that the surge period has ended, operations are the same as 
previously described when there are no surge requirements. When OptempoManager is 
notified that the surge period has ended, the number of required coverage hours becomes the 
required number of coverage hours for the period following surge operations. Upon achieving 
this requirement, the OptempoManager notifies UAVSim to egress all UAVs just as 
previously discussed. The number of hours required for the UAV company to achieve the 
limited coverage is reconied and when all UAVs dave completed maintenance, the company's 


down time is also recorded. 


38 





EndSurgePeriod 





Figure 10: SurgeManager Event Graph 


4. Sensor Package Failures. 

The need for sensor package failures was added because of specifications listed in the 
ORD. These failures are considered to be MAFs. The time at which a sensor package failure 
occurs is determined by drawing a random number -from the sensor package failure 
distribution. This time starts at launch; a MAF can only occur during flight. Given that both 
platform and sensor package failures are MAFs, the point at which a UAV becomes non- 
mission capable is determined by finding the minimum of a UAV's platform MAF time and 
sensor package failure time. If the cause of the UAV's MAF was the sensor package failing, 
the length of maintenance time is drawn from the Repair Sensor MAF distribution. 

5. Maintenance Prioritization. 

When maintenance prioritization is used, UAVs enter a priority queue after landing. 
The priority of this queue is determined by required maintenance times where lower 


maintenance times have higher priority. Prioritization 1s not preemptive. 
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6. Enhanced Maintenance System. 

The maintenance portion of the model was expanded so that the appropriate type of 
maintenance and the corresponding maintenance time could be assigned. Two types of 
maintenance are modeled: corrective and preventive. Corrective maintenance consists of all 
maintenance actions as a result of a MAF plus any scheduled maintenance that is due. 
Preventive maintenance is all maintenance necessary to sustain the UAV. The goal of 
preventive maintenance is to retain the UAV at a certain level of performance [Ref. 25:p. 48]. 

When each UAV enters the maintenance phase, a determination of whether a MAF 
occurred is made. If a MAF occurred, the UAV enters corrective maintenance. Otherwise, 
the UAV enters preventive maintenance. When corrective maintenance must be performed, a 
determination of whether or not a service is due and the cause of the MAF is made. Ifa 
service is due, the required maintenance time is the sum of the scheduled service time, 
logistics delay time, non-MAF repair time, and the time to repair the MAF. Else, the required 
maintenance time is the same sum less the scheduled service time. If the UAV is to undergo 
preventive maintenance, the requirement for a service is also determined. Should the UAV 
require a service, the maintenance time is the scheduled maintenance time, plus the logistics 
delay and non-MAF times. Otherwise, the maintenance time is the sum of the preventive 
maintenance time, logistics delay time and non-MAF repair time. Figure 11 shows this 


maintenance flow. 
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Maintenance Phase 


MAF No MAF 
(corrective maintenance) (preventive maintenance) 


service due no service due service due ho service due 
(constant time) {constant time) (RoutinePreventiveM aintenanceTime) 
sensor MAF platform M AF 
(RepairSensorMAFTime) | j(RepairPlatormM AFTime) 


Figure 11: Enhanced Maintenance System Flow 


7. Non-Perfect Maintenance. 

Imperfect maintenance was modeled by allowing the UAV to "remember" after how 
many flight hours it would experience a MAF. This is an option in the model. Each time a 
UAV enters preventive maintenance, under this option the flight time is subtracted from the 
time to occurrence of the next scheduled MAF. Thus, the UAV would continue to fly until it 
had the assigned MAF. This negates the "good as new" assumption. 

8. Specification of Distributions. 

The analyst has the option to specify the distribution and corresponding parameters for 
each of the distributions used in the model. This feature was added so analysts could perform 
analysis without being restricted to the exponential distribution, as it may not always be 
appropriate. 

J. EXPANDED MODEL ASSUMPTIONS. 

Given the added features of the model, not all of the original assumptions are 
necessary. Referring to Table 2, original assumptions 5 and 7 are no longer needed. The © 
initial assumption of perfect maintenance can be optionally eliminated. Crew-related 


limitations are added with the addition of the GCSChief and MaintenanceChief components. 


Al 


Now the number of personnel available to work and the number of hours that they can work 
are limiting factors. 
K. EXPANDED MODEL INPUTS. 

The expanded version of UAVSim has the same inputs as previously discussed with 
the addition of the new inputs listed in Table 12. The inputs which were added are those 
required to make the model more robust and allow for analysis of requirements specific to the 
TUAV. Specifically, TUAV systems typically do not have the endurance, number of 
platforms, or number of personnel to perform extended continuous operations. 


Prioritize Maintenance | The user can specify whether or not a 
priority queue is used when determining 


which UAV is serviced next. 

Wear and Tear Allows the user to specify whether or not 
UAVs are "as good as new" once they are 
serviced. 


Optempo (hours) | The requirement for a UAV to achieve a 
specified number of coverage hours per 
day. 

Allows the user to indicate whether or not 
there is a requirement for non-continuous 
operations. 


Surge Constraints Allows the user to indicate whether there 
"surge" for a specified length of time. 
will perform surge operations. 


Surge Optempo (hours) The limited amount of time that a UAV 
company must provide coverage before 
returning to optempo requirements. 

Maintenance Constraints Indicates if UAV company operations will 
be limited by maintenance crew-related 
limitations. 


Taal 
maintenance team can work per day. 

GCS Constraints | Indicates if the UAV company operations 

will be limited by GCS crew-related 


limitations. 
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The following times require distributions 
and the corresponding parameters: 

Time to Platform MAF 

Time to Sensor MAF | 

Repair Time for Sensor MAF 

Repair Time for Platform MAF 
Preventive Maintenance Time 
Logistics Delay Time 

A brief description of each of these 
random times is given in Table 14. 


Distributions and Parameters 


















There are several distributions available in 
UAVSim. A listing of the available 
distributions and the required parameters 
are listed in Table 15. Because of the 
design of UAVSim, the analyst can also 
code and add other distributions as well. 
For example, for the purposes of this 
thesis, the Weibull distribution was coded 
and added to UAVSim. The time to a 
non-MAF is always modeled as iid 
exponential and thus only requires a 















| mean time between non-MAFs. 
: to reach steady-state. 
Relative Precision The relative precision desired. The sample 






size for each run of the simulation was 
determined using relative precision. This 
concept is to make replications of the 
simulation until the half-length of the 
confidence interval divided by the mean 
value of the MOP is less than or equal to 
the desired relative error [Ref 26:p. 537]. 











Table 12: Expanded Model Inputs 


Each of the random times is defined in the following table. The observations for each 


of these times are drawn from distributions that the analyst selects. 
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The time to the occurrence of a platform 
related MAF. 


The time to the occurrence of a sensor 
package related MAF. 


Time to Platform MAF 


Time to Sensor MAF 




















The corrective maintenance time required to 
repair all deficiencies caused by a sensor 
MAF. 
The corrective maintenance time required to 
repair all deficiencies caused by a platform 
MAF. , 

The preventive maintenance time required to 
sustain the performance of the UAV. The 
mean of these observations is the mean 
preventive maintenance time (Mpt) or mean 
time to repair (MTTR). : 
The maintenance downtime as a result of 
waiting for spare parts, waiting for 
availability of equipment, etc. 


Repair Time for Sensor MAF 


















Repair Time for Platform MAF 











Preventive Maintenance Time 




















Logistics Delay Time (LDT) 






Table 13: Description of Each Random Time 


The following table lists the distributions that are available in UAVSim and the 


required parameters for each. 
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Table 14: Distributions Available in UAVSim 


The scconeatial beta, gamma, and normal distributions are already implemented in 
Sumkit. However, the Weibull and lognormal distributions are not in Simkit and had to be 
coded by the author. Refer to APPENDIX C for an explanation of the algorithms used to 
implement these distributions as well as verification of their results. 

L. | EXPANDED MODEL OUTPUTS. 

The expanded version of UAVSim requires additional MOPs for analysis. Some of 
the previous MOPs from the MASS model and first version of UAVSim are not appropriate 
for analysis of TUAVs. For example, when TUAVs are performing non-continuous 
operations, ETOS is not a suitable MOP. A more fitting MOP is the expected number of 
hours required to achieve the commander's requirement for coverage of the AO. For example, 
if ee commander specifies that he wishes to have twelve hours of continuous coverage and it 


takes an average of thirteen hours to achieve that requirement, then thirteen hours is the 
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expected number of hours required to achieve the requirement. The additional MOPs 
generated by the model are listed in Table 15. 
Also, an example and explanation of the output files generated by UAVSim is 


presented in APPENDIX A. 






The percentage of days that the UAV 
company is able to achieve the required hours 
of coverage. | 
Given that the UAV company is able to 
achieve the required OPTEMPO, this is the 
average amount of time that was required for 
the UAV company to meet the requirement. 
Given that a UAV company achieved the 
required OPTEMPO, and all of the UAVs 
have completed maintenance on the same day 
that the OPTEMPO requirement was met, 
down time is the remaining hours of the day 
in which the company does not fly nor 
service UAVs. 
The percentage of days in which the company 
is able to achieve the specified limited 
coverage requirement following a surge 
period. 
The number of hours necessary for the. 
company to achieve the limited coverage 
requirement following a surge period. 
Once the company achieves the limited 
coverage requirement and all UAVs have 
been serviced, down time is the remaining 
part of the day in which the company does 
not fly UAVs nor perform maintenance. 
Mean Number of Sorties The mean number of sorties flown during a 
deployment. 


Sortie Generation Rate (sorties/day) The number of sorties flown per day. 


Mean Wait Time in Queue The mean time UAVs spend in the 
maintenance queue prior to being serviced. 


Percent Days OPTEMPO Achieved 

















Hours to Achieve OPTEMPO 


Company Down Time 


Percent Days Surge OPTEMPO Achieved 




























Time to Achieve Surge OPTEMPO 





















Company Surge Down Time 











Mean NMC Time The average amount of time that a UAV is 
non-mission capable prior to being serviced. 

Scheduled Service Ratio Percentage of time a UAV is required to have 
a scheduled service. 


Table 15: Expanded Model Outputs 
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Given the additional analytic capabilities provided by this version of UAVSim, it is 
now possible to explore the performance of TUAV systems by varying the input parameters. 
Additionally, the analyst can gain ideas about the sensitivity of TUAV performance with 


respect to changes in input parameters. These explorations are the focus of Chapter IV. 
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IV. ANALYSIS AND RESULTS. 





A. GENERAL. 

As mentioned in Chapter III, UAVSim involves numerous stochastic processes. Such 
processes can exhibit two types of behavior: transient or steady-state. Transient behavior is 
indicative of early or erratic operations during which the observations are more biased toward 
initial conditions whereas steady-state observations are not. Theoretically, steady-state is 
reached 1n the limit as time approaches infinity; however, there is a point in finite time where 
it can be assumed that a system is in steady-state. UAVSim can be used to perform both 
transient and steady-state analyses. An examination of results using steady-state will be 
performed in this chapter. The discussions presented show some but not all of the capabilities 
of UAVSim. 

B. DETERMINATION OF STEADY-STATE. 

In order to perform steady-state analysis, two questions have to be answered: "When 
| does the system enter steady-state?" and "After how many runs should the simulation be 
terminated?" Determining steady-state is important in analysis of the MOPs because steady 
state is the point at which the observations of the MOP are no longer biased by the initial 
conditions. 


We desire to provide estimates with a prescribed degree of accuracy. The objective is 





to obtain an accurate estimate of the true mean and an accurate confidence interval that covers 
| the true mean [Ref. 26:p. 538]. To do such requires knowledge of how many replications or 


number of deployments that must be conducted in order to obtain a specified error in the 





estimate of the mean [Ref. 26:p. 536]. There are two methods to determine when to terminate 
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a simulation, absolute sealin and relative precision. In this study, relative precision will be 
used and is entered as an input parameter. 

Because of the options available in UAVSim, it was necessary to determine steady- 
state for both the continuous and non-continuous coverage cases. The method used was to 
graph the MOPs for different deployment lengths and by inspection determine the point at 
which the values of the MOP appear to "settle down." 

For the continuous case, the primary MOP was ETOS. With inputs given in Table 5, 
and varying the length of deployment, the simulation appeared to reach steady-state after 230 
days. Figure 12 shows a graph of simulation length vs. the value of ETOS. For this case, we 


will "warm up" the simulation for 230 days prior to collecting out steady-state output. 


Steady-State for ETOS 





0 50 100 150 200 250 300 
‘Simulation Length (days) 


Figure 12: ETOS Steady-State 
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In the non-continuous case, there are two situations that could be studied. The first 
case (Case [) is when OPTEMPO constraints are active and the second case (Case II) is when 
surge constraints are active. In both cases, the MOP that required the longest simulation 
length to reach steady-state or “down time." The length of a simulation run was determined 
to be 360 and 450 respectively for Case I and Case II using inputs in Table 5. Figures 13 and 


14 show the graphs of the MOPs vs. the length of simulation. 


Steady-State for OPTEMPO Down Time 
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Figure 13: Steady-State for Optempo Down Time 
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Figure 14: Steady-State for End Surge Period Down Time 


C. CONTINUOUS AND NON-CONTINUOUS OPERATIONS. 

UAVSim allows the analyst to examine continuous and non-continuous operations for 
UAVs. In this chapter both topics will be explored and the reader should note the context in 
which each applies. Continuous operations will be assumed to be operations at an Army 
division and higher level while non-continuous will be associated with brigade and below 

| operations. The current version of the model is flexible enough to also examine operations at 

the Corps and higher levels. However, the focus of this study will be at division and brigade 
levels. 
D. DIVISION OPERATIONS. 

Interviews with members of the division intelligence staff at the 4" Infantry Division, 


Fort Hood, Texas, indicate that there is a requirement at the division level to have continuous 
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coverage of an AO. There are several limiting factors which may affect the ability of a UAV 
company to provide such support, one of which is positioning. It is desired to position the 
company as far away from areas of hostile activity as possible for the purposes of force 
protection and still fulfill requirements. New weapons systems such as the Paladin artillery 
system and follow-ons, i.e. - Crusader, will allow the division to engage the enemy at much 
greater ranges. An immediate question is "What is the effective range of a UAV company?" 
The approach taken to answer this question was to vary positioning of a UAV system and 
compare the values for ETOS. 

1. Positioning of a UAV Company. 

Consider a scenario in which a division commander desires to have continuous 
coverage of an AO. The operations and intelligence sections of the division staff must 
determine where to position the UAV company so that it can effectively perform its mission. . 
It seems logical that for limited UAV endurance, the value of ETOS would decrease as the 
ingress time increases (distance to the AO). This is synonymous with varying the distance 
from the launch and recovery site to the AO. Such analysis will provide the decision-maker 
with an indication of the proportion of time that a company of UAVs will cover the AO. The 
inputs for this analysis for a one hour ners time are presented in Table 16. All of the 
distributions are assumed to be exponential. 

Figure 15 shows the relationship between ingress time and ETOS. As the ingress time 
increases, the value of ETOS eens UAVs have a longer distance to fly in order to reach 
the AO and thus spend less time on station. This would be a planning consideration for staffs 
allocating UAVs for missions. ETOS would indicate the proportion of time that a company's 


baseline of UAVs could provide RSTA. For this scenario, positioning a UAV system so that 
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ingress time is about one hour would result in approximately 90% effectiveness; however, a 
commander requiring a greater percentage of coverage time may position the company thirty 


minutes from the AO. 


SIMULATION Input data for \code\test1\lie20maf.txt.out: 

Number of platforms: 4 

Number of maintenance paths: 1 

Priority maintenance active: false 

Wear and tear allowed on UAVs: false 

Maintenance constraints active: false 

Number of maintenance teams: 1 

Maximum number of platforms in flight :2 

GCS constraints active: false 

Number of GCS crews: 1 

Length of Warm Up Period (days): 230.0000 

Planned Length of Deployment (days): 30.0000 

Actual Length of Simulation (days): 260.0000 

OPTEMPO constraints active: false 

Deployment OPTEMPO (hours): 12.0000 

Surge OPTEMPO constraints active: false 

Surge OPTEMPO (hours): 8.0000 

Ingress time (hours): 1.0000 

Egress time (hours): 1.0000 

Scheduled’on station time (hours): 4.0000 

Platform turn time (hours): 0.5000 

Time to complete scheduled maintenance (hours): 7.0000 

Mean repair time for each non-mission affecting failures (hours): 0.5000 
Flight time between scheduled maintenance actions (hours): 50.0000 
Mean time between non-mission affecting failures (hours): 5.0000 
The z value used to calculate confidence intervals: 1.9600 


DISTRIBUTIONS and Parameters: 

Platform MAF Time: Exponential 

Platform MAF Time Parameters: 20.0 

Sensor Package MAF Time: Exponential 
Sensor Package MAF Time Parameters: 110.0 
Platform Repair Time: Exponential 
Platform Repair Time Parameters: 2.0 

, Sensor Package Repair Time: Exponential 
Sensor Package Repair Time Parameters: 2.0 
Logistics Delay Time: Exponential 
Logistics Delay Time Parameters: 0.5 
Preventive Maintenance Time: Exponential 
Preventive Maintenance Time Parameters: 0.5 


Table 16: Inputs for Effect of Positioning on ETOS 
There is a discussion of these inputs in Table 12. 
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ETOS as a Function of Ingress Time 


ETOS 





0.5 1.0 1.5 2.0 2.5 
Ingress Time (hrs) 


Figure 15: The Effect of Positioning on ETOS 
The UAVs in this example each had a fixed endurance of 4 hours. 

2. Endurance. 

The previous example was a particular UAV System with a fixed endurance. Next, 
we examine multiple systems with differences in endurance. For illustrative purposes, assume 
that the goal is to achieve a 95% ETOS. We fix the ingress time at one hour and run the model 
with UAVs having different endurance capabilities. : The inputs are the same as those shown 
in Table 16 with the exception that in each run of the model, a UAV system with a different 
endurance is used. Figure 16 shows the result of this analysis. A UAV system consisting of 


four platforms with an endurance of eight hours satisfies the goal. 
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ETOS as a Function of Endurance 





UAV Endurance (hrs) 


Figure 16: Systems with Different Endurance 
The UAVs in this example each had a fixed ingress time of 1 hour. 
3. The Benefit of Maintenance Prioritization. 

There may be alternatives less costly than increasing the endurance of a UAV system. 
One alternative is to change the company maintenance policy. In a study conducted by Post 
and Warner, one of the questions that could not be answered using the MASS model was 
"Does priority maintenance optimize ETOS?" [Ref. 17]. Maintenance prioritization has been 
added as an option to UAVSim and now it is possible to address this issue. The same inputs 
used in the previous example, Table 16, are used here with the exception that now 
maintenance is prioritized in an effort to improve ETOS. Figure 17 shows a comparison of 


ETOS as a function of priority and non-priority maintenance. 
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Priority vs Non-Priority Maintenance 


EJ Priority Maint 
No Priority Maint 





Ingress Time (hrs) 2° 
Figure 17: Benefit of Priority Maintenance 


There is no noticeable benefit associated with using priority maintenance. This result 
is similar to that proposed in "Operational Analysis of Sustainability of a Mobile Military 
Platform" [Ref. 27:p. 23]. In that study, the effect of maintenance repair times on ETOS was 
examined. From that comparison, it was hypothesized that ETOS was not sensitive to 
changes in the company's maintenance structure. These findings support that hypothesis. 
These findings suggest that money and/or other resources may be better used by not investing 
in prioritizing maintenance. One possible excianaien for this lack of sensitivity to 
prioritization is that UAVs typically spend a relatively short amount of time in maintenance or 
waiting for maintenance. For a one hour ingress time, the mean wait time for maintenance 
was 1.7914 hours and 1.5950 hours for non-priority and priority maintenance respectively. 
This only resulted in a 10.96% or 12 minute decrease in wait time. In relation to the total 


cycle time, a decrease of 12 minutes does not make a practically significant difference. 
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4. Number of Maintenance Paths. 

It seems plausible that increasing the number of maintenance paths would decrease the 
amount of tine that UAVs are not able to fly and thus increase the value of ETOS. The effect 
of increasing the number of maintenance paths will be explored here. The method used was to 
vary the number of available maintenance paths from one to four while keeping all other 
factors constant (in particular, 4 UAVs) and evaluating the resulting values of ETOS. Each . 
maintenance path had its own maintenance crew with no limitations. The following figure 


illustrates the benefit associated with increasing the number of maintenance paths. 


Benefit of Additional Maintenance Paths 


ETOS 





7) 
0 1 3 9 3 4 5 


Number of Maintenance Paths 
Figure 18: Number of Maintenance Paths 
The number of UAVs was held constant at 4 in this example. Each 
maintenance path did not have crew-related limitations. 


The gain associated with increasing from one to two maintenance paths is significant. 


However, additional maintenance paths do not improve ETOS. 
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E. DISTRIBUTIONS AND ETOS. 


Thus far, explorations have been performed with the assumption that all distributions 
within the model! are exponential. But, what if the distributions are not exponential? How 
does the distribution affect the value of ETOS and is the model sensitive to changes in the 
distributions? 

UAVSim is very flexible in that the analyst can specify the distribution for any of the 
random times. Not only can any distribution that already exists in the model be used but 
distributions can be created and added as well. A major point is that the model does not have 
to be modified or recompiled in order to use or add a different distribution; moreover, the 
model remains unchanged. This is a major benefit of using a language such as Java and the 
component approach for modeling and simulation. With such a feature, the user is much 
better equipped to perform “what if “ analyses. 

To demonstrate this ability, Sculptured (a), Sculptured (b), and Triangular distributions 
were implemented and used in a comparison with the exponential and Weibull. We chose 
these to illustrate the flexibility of VAVSim, and not because they are necessarily UAV failure 
distributions. 

1. Sculptured(a) and Sculptured(b) Distributions. 

Both of the Sculptured distributions were developed and explained in "Distribution 
Sculpturing or Inverse Modification" [Ref. 28:pp. 36-44]. These distributions are derived 
from the exponential distribution but have different effects. They produce more short MAF 
times than the exponential although the mean 1s the same as the exponential. This results in a 


heavier night hand tail. 
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The algorithm coded and implemented for the Sculptured distributions is presented in 
APPENDIX E. Even though the algorithm was from a published source, we verified its 
implementation. The method used to verify that these two distributions were producing 
correct results was to first generate the value of the parameter of the distribution so as to have 
the same mean as the exponential. The value of MTBMAF used was the objective value, 20 
hours. Next, 1000 random variates were generated and a histogram of those observations 
created. The mean of the empirical data was calculated and then compared with the true 
mean. 

Table 17 shows the values of the parameters the Sculptured and other distributions that 
were used in this comparison. The reader should note that all distributions were parameterized 
SO that the theoretical mean would be the same. The exponential, Sculptured(a) and 
Sculptured(b) are one parameter distributions. Once a mean is selected, the user has no 


control over the variance. 


Mean Variance 


Table 17: Parameters for Distributions 
















The reader should note that for both Sculptured distributions, the empirical mean is 
very. close to the theoretical. An exploration in which 10,000 and 20,000 variates were 
generated and the empirical mean calculated confirmed that there is only a small difference in 


the theoretical and empirical means. 
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2. Triangular Distribution. 

The Tnangular distribution was created and used in this analysis because it is often 
used for a rough estimate where there is a limited amount of data [Ref. 26:p. 343]. There are 
three basic types of the triangular distribution: right, left, and general. All three of these 
distributions were added and are available in UAVSim; however, the general form of the 
distribution can be parameterized such that it can generate variates for all types. In this 
comparison, it was hypothesized that the Right Triangular would be appropriate. 

3. Results and Comparison. 

Table 18 shows the values of the input parameters for the Sculptured(b) MTBMAF, 
one of these cases used in this comparison. The only variation for the data presented is that 


the MAF distribution and parameters were changed for each run. 
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SIMULATION Input data for \code\test14\sculpB.txt.out: 

Number of platforms: 4 

Number of maintenance paths: 1 

Priority maintenance active: true 

Wear and tear allowed on UAVs: false 

Preventive Maintenance Effectiveness: 0.0 

Maintenance constraints active: false 

Number of maintenance teams: 1 

Maximum number of platforms in flight :2 

GCS constraints active: false 

Number of GCS crews: 1 

Length of Warm Up Period (days): 230.0000 

Planned Length of Deployment (days): 30.0000 

Actual Length of Simulation (days): 260.0000 

OPTEMPO constraints active: false 

Deployment OPTEMPO (hours): 12.0000 

Surge OPTEMPO constraints active: false 

Surge OPTEMPO (hours): 8.0000 

Ingress time (hours): 1.0000 

Egress time (hours): 1.0000 

Scheduled on station time (hours): 4.0000 

Platform turn time (hours): 0.5000 

Time to complete scheduled maintenance (hours): 7.0000 

Mean repair time for each non-mission affecting failures (hours): 0.5000 
Flight time between scheduled maintenance actions (hours): 50.0000 
Mean time between non-mission affecting failures (hours): 5.0000 
The z value used to calculate confidence intervals: 1.9600 


DISTRIBUTIONS and Parameters: 

Platform MAF Time: SculpturedB 

Platform MAF Time Parameters: 0.7917 

Sensor Package MAF Time: Exponential 

Sensor Package MAF Time Parameters: 110.0 

Platform Repair Time: LogNormal 

Platform Repair Time Parameters: 0.6616225 0.2510964 
Sensor Package Repair Time: LogNormal 

Sensor Package Repair Time Parameters: 0.6616255 0.2510964 
Logistics Delay Time: LogNormal 

Logistics Delay Time Parameters: -0.7246719 0.2510964 
Preventive Maintenance Time: LogNormal 

Preventive Maintenance Time Parameters: -0.6983266 0.107785 


Table 18: Input Parameters for Comparison. 


Table 19 shows the mean values for ETOS and the 95% confidence intervals. Figure 


19 illustrates the differences in the observed values of ETOS for each of the distributions. 
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Distribution | Mean ETOS 
ETOS | 95% Confidence Interval 


Table 19: Results of Comparison 
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Figure 19: The Effect of Various MAF Distributions 


This demonstrates that the user can select or create any distribution desired for 

, analysis. The distribution assumed has a significant impact on the resulting value of ETOS. 
In this example, only the first moment for each of the distributions could be matched. The 
empirical mean and variance are shown in Table 19. The empirical means are relatively close. 
However, there are large differences in the empirical variances. If it were possible to match the 
first and second moments of these distributions, a stronger statement about the relationship 
between the distribution and ETOS could be made. It is recommended that during the testing 


of UAVs, data on the failure times be collected so that statisticians can determine the true 
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distribution of MAFs. Decisions made without a good approximation of the true distribution 
may result in drastic overestimates or underestimates. For example, decisions based on 
analysis assuming that the MAF times are exponential result in approximately a 16% 
overestimate if the true distribution were the Weibull of our example. The effect for the 
division commander is that approximately 16% of the time when he expected to have 
coverage of the AO, he would not. 

F. VARIANCE OF TIME TO MAF. 

The ORD for the brigade commander's UAV lists several objective and threshold 
values for a proposed system. The value that is listed is a mean or average. Yet, is 
specification of the mean value enough? It is hypothesized that only specifying the mean is 
not enough. A stipulation on variance should also be given. For cases in which the time to a 
MAF fits the exponential distribution or another one-parameter distribution, a mean is 
sufficient. Knowing the mean for a one-parameter distribution such as the exponential implies 
knowledge of the variance. However, if the appropriate distribution is not the exponential but 
a two-parameter distribution such as the Weibull, then specification of only the mean is not 
enough. This is an important issue since engineers may design a UAV system to meet a 
certain MTBMAF, but a large variance may have significant effects on performance. 

The Weibull distribution is often used to model times to failure for mechanical 
equipment [Ref. 26:p. 333]. For the next examples, we assume the MAF time follows the 
Weibull distribution. The Weibull requires two parameters: the scale parameter that will be 
referred to as B and a shape parameter a. If the distribution of platform MAF times is 
Weibull, significant differences in results can occur compared to the exponential distribution 


even though the mean is equal to that specified in the ORD. 








1. Values of o and B. 


Values for the parameters of a Weibull distribution with a mean equal fo the objective 





and threshold values, 20 and 54 hours respectively, were calculated. The values for a and B 
are calculated for increasing hazard rates (a > 1) and decreasing hazard rates (a < 1). Graphs 
of the distributions are shown in APPENDIX D. The Weibull with a = 1 is an exponential 


with A = 1/8. 


[Mean [a | _B | Mean (hrs) | Variance (hrs) 
Threshold, 20 br 
3537 ; 























Table 20: Moments for Various Weibull Distributions 


Objective, 54 hrs 





2. ETOS Sensitivity to Variance. 
To demonstrate the effect of IFR and DFR on ETOS, the scenario previously 
discussed will be revisited. The performance of a UAV system was examined using four 


possible failure rates. The inputs for the simulation are shown in Table 21. The reader should 





note that the only input that was changed on each run was the vector of parameters for the 
platform MAF. Four values for a and the corresponding B's were calculated and used. These 
values yield theoretical means that are the same as specified in the ORD. The ETOS was 


determined so that the commander could have an indication of the percentage of time that the 





UAV company could provide RSTA. 
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SIMULATION Input data for \code\test3\pt5-plusMinus5.txt.out: 
Number of platforms: 4 

Number of maintenance paths: 1 

Priority maintenance active: true 

Wear and tear allowed on UAVs: false 

Preventive Maintenance Effectiveness: 0.0 

Maintenance constraints active: false 

Number of maintenance teams: 1 

Maximum number of platforms in flight :2 

GCS constraints active: false 

Number of GCS crews: 1 

Length of Warm Up Period (days): 230.0000 

Planned Length of Deployment (days): 30.0000 

Actual Length of Simulation (days): 260.0000 

OPTEMPO constraints active: false 

Deployment OPTEMPO (hours): 12.0000 

Surge OPTEMPO constraints active: false 

Surge OPTEMPO (hours): 8.0000 

Ingress time (hours): 1.0000 

Egress time (hours): 1.0000 

Scheduled on station time (hours): 4.0000 

Time to complete scheduled maintenance (hours): 7.0000 

Mean repair time for each non-mission affecting failures (hours): 0.5000 
Flight time between scheduled maintenance actions (hours): 50.0000 
Mean time between non-mission affecting failures (hours): 5.0000 
The z value used to calculate confidence intervals: 1.9600 


DISTRIBUTIONS and Parameters: 

Platform MAF Time: Weibull 

Platform MAF Time Parameters: 0.25 0.8333 

Sensor Package MAF Time: Exponential 

Sensor Package MAF Time Parameters: 110.0 

Platform Repair Time: LogNormal 

Platform Repair Time Parameters: 0.6616225 0.2510964 
Sensor Package Repair Time: LogNormal 

Sensor Package Repair Time Parameters: 0.6616255 0.2510964 
Logistics Delay Time: LogNormal 

Logistics Delay Time Parameters: -0.7246719 0.2510964 
Preventive Maintenance Time: LogNormal 

Preventive Maintenance Time Parameters: -0.6983266 0.107785 


Table 21: Input Values for Scenario 
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The values of ETOS are shown in Table 22 along with a 95% confidence interval. 






Variance ) | ETOS 95% Confidence Interval 


Mean (hrs) |___95% Confidence Interval _ 
27600 0.3921 0.3460 - 0.4383 
0.8770 0.8629 - 0.8912 









0.9536 0.9416 - 0.9656 
0.9757 0.9663 - 0.9851 


Table 22: Parameters for Weibull 





The variance of the time to a MAF does effect the value of ETOS, even though the - 
mean is held constant and the same distribution (Weibull) was used. As the variance 
| decreased, the associated ETOS increased because the platforms flew longer without a MAF. 
Thus the company is better able to meet the commander's requirement. We conclude that 
specification of a MTBMAF may not be sufficient and that the ORD should also specify a 
variance of time between mission affecting failures. | 
G. INCREASING THE MTBMAF. 

A comparison can also be performed using the threshold value for MTBMAF. The 
inputs to the model were kept the same with the exception of the parameters for the Weibull 
distribution. Figure 20 shows this comparison and also shows the benefit of increasing 
MTBMAF from the threshold to the objective value. As was expected, the higher MTBMAF 
would increase the company's ability to support the commander. However, there does not 


appear to be a significant increase. 
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Comparison of MTBMAFs 
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Figure 20: Comparison of Values for MTBMAF 


H. EFFECT OF NON-PERFECT MAINTENANCE. 

Thus far, analysis has been performed assuming that the maintenance section will 
perform "perfect maintenance." This assumption is unrealistic. It is quite possible and 
probable that a maintenance crew will not discover and fix every deficiency on a piece of 
equipment. In other words, "non-perfect maintenance" will exist. A discussion of how this 
type of maintenance was modeled is presented in Chapter III. The inputs for this comparison 
are the same as those given in Table 21 with the exception that the option "Wear and tear 


allowed on UAVs" was set to true. The results of this comparison are shown in Figure 21. 
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UAV Non Perfect Maintenance 
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Figure 21: Effect of Non-Perfect Maintenance 


Allowing nope perteet maintenance does decrease performance; however, there is not a 
drastic difference in performance. 
I. “BRIGADE OPERATIONS. 

Thus far, the analysis has focused on division operations with continuous operation of 
a UAV company. Now the focus will be shifted to brigade operations. The brigade has fewer 


UAV assets than were available at the division level primarily because the AO for a brigade is 





approximately one-third of a division's. The requirement for UAV support at this level is 
projected to be twelve hours of continuous coverage versus the twenty-four. Initially, the 
reliability of the UAV will be examined and the remaining analysis will be on the structure of 
the company. ETOS is no longer the most appropriate MOP. The expected number of hours 


to achieve OPTEMPO will be evaluated. 
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Thus far in this study, the value for sensor package MTBMAF has been the threshold 
value, 110 hours. However, as with other parameters in the ORD, an objective value is 
presented as well. This value for sensor MTBMAF is 160 hours. A question of interest might 
be: "Is there a benefit associated with increasing the MTBMAF for sensors?" The exponential 
distribution is often used to model the time to failure for electronic equipment and is used here’ 
as well. 

We run the simulation with platforms having sensor MTBMAFs of 110 and 160 hours 
and compare hours to achieve OPTEMPO. The inputs are shown in Table 23. All inputs 
were held constant with the exception of the parameter for the sensor package failure 


distribution. Figure 22 shows the results. 
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SIMULATION Input data for \code\test12\lpt5-mtbmaf160.txt.out: 
Number of platforms: 4 

Number of maintenance paths: 1 

Priority maintenance active: true 

Wear and tear allowed on UAVs: true 

Preventive Maintenance Effectiveness: 0.0 

Maintenance constraints active: false 

Number of maintenance teams: 1 

Maximum number of platforms in flight :2 

GCS constraints active: false 

Number of GCS crews: 1 

Length of Warm Up Period (days): 360.0000 

Planned Length of Deployment (days): 30.0000 

Actual Length of Simulation (days): 390.0000 

OPTEMPO constraints active: true 

Deployment OPTEMPO (hours): 12.0000 

Surge OPTEMPO constraints active: false 

Surge OPTEMPO (hours): 8.0000 

Ingress time (hours): 1.5000 

Egress time (hours): 1.5000 

Scheduled on station time (hours): 3.0000 

Platform turn time (hours): 0.5000 

Time to complete scheduled maintenance (hours): 7.0000 

Mean repair time for each non-mission affecting failures (hours): 0.5000 
Flight time between scheduled maintenance actions (hours): 50.0000 
Mean time between non-mission affecting failures (hours): 5.0000 
The z value used to calculate confidence intervals: 1.9600 


‘DISTRIBUTIONS and Parameters: 

Platform MAF Time: Weibull 

Platform MAF Time Parameters: 0.75 16.7977 

Sensor Package MAF Time: Exponential 

Sensor Package MAF Time Parameters: 160.0 

Platform Repair Time: LogNormal | 

Platform Repair Time Parameters: 0.6616225 0.2510964 
Sensor Package Repair Time: LogNormal 

Sensor Package Repair Time Parameters: 0.6616255 0.2510964 
Logistics Delay Time: LogNormal 

Logistics Delay Time Parameters: -0.7246719 0.2510964 
Preventive Maintenance Time: LogNormal 

Preventive Maintenance Time Parameters: -0.6983266 0.107785 


Table 23: Input Data for Sensor MTBMAF Comparison 
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Sensor MTBMAF Comparison 


Mi MTBMAF = 110 
COMTBMAF = 160 


Time Achieve OPTEMPO 





Ingress Time (hrs) 


Figure 22: Benefit of Increasing Sensor MTBMAF 


By increasing the MTBMAF from 110 to 160 hours, there is no noticeable difference 
in time to meet the OPTEMPO requirement. This analysis indicates that sensor MTBMAF 
may have very little or no effect on fulfilling the coverage requirement. One possible 
explanation is that because the MTBMAF for platforms is so low, that sensor failures rarely 
affect performance of the UAV. 

J. BRIGADE UAV COMPANY STRUCTURE. 

"How many UAVs are enough?" There is no one correct answer to this question. In 
brigade operations, this number is driven by at least two factors: OPTEMPO requirement and 
endurance. This suggests that the best UAV system should be flexible enough to handle the 


range of operations that a brigade UAV company is expected to face. 
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1. Number of UAVs. 


The first analysis will be an exploration into the number of UAVs required to fulfill a 
commander's requirement to maintain a 12-hour OPTEMPO. Consider a scenario in which a 
brigade commander desires to have 12 hours per day of continuous coverage. How many 
UAVs does he need? We present an example of using UAVSim to determine a possible 
solution. 

Runs of the simulation were conducted holding OPTEMPO constant; however, the 
number of UAVs employed ranges from 1 to 6 each. The MOP examined was hours to 
achieve OPTEMPO. There is a startup cost associated with meeting an OPTEMPO, the 
ingress time. The objective is to provide coverage with the isi time required to do so being 
as close as possible to the sum of the ingress time and the coverage requirement. The 
following figure shows the relationship between the number of UAVs and the number of 


hours required to meet a twelve-hour OPTEMPO. 
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Number UAVs vs Achieving OPTEMPO 


Time to Achieve Optempo (hrs) 





Number UAVs 


Figure 23: UAVs to Achieve OPTEMPO 


Figure 23 indicates that for a twelve hour OPTEMPO, at most four UAVs would be 
needed. After four UAVs there is very little if any gain. | 

2. UAV Endurance. 

The second factor mentioned as a factor in determining the number of UAVs was 
endurance of the platforms. A platform cannot fly forever. Limiting factors include the need 
for fuel and time intervals between maintenance. As such, UAVs have a set endurance that is 
defined as the amount of time that a UAV can fly between launch and landing. It may seem 
that more is better, but the real question is: "How much is enough?" The following example 
demonstrates the ability to use UAVSim as an analysis tool to try to get an answer. 

Runs of the simulation were performed in which all input parameters were the same 
with the exception of endurance; the sum of the planned ingress time, time on station, and 


egress time remained constant. The input parameters are shown in Table 24. 
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SIMULATION Input data for \code\test9\5endurance2uav.txt.out: 
Number of platforms: 2 

Number of maintenance paths: 1 

Priority maintenance active: true 

Wear and tear allowed on UAVs: true 

Preventive Maintenance Effectiveness: 0.0 

Maintenance constraints active: false 

Number of maintenance teams: 1 

Maximum number of platforms in flight :2 

GCS constraints active: false 

Number of GCS crews: 1 

Length of Warm Up Period (days): 360.0000 

Planned Length of Deployment (days): 30.0000 

Actual Length of Simulation (days): 390.0000 

OPTEMPO constraints active: true 

Deployment OPTEMPO (hours): 12.0000 

Surge OPTEMPO constraints active: false 

Surge OPTEMPO (hours): 8.0000 

Number of simulated deployments: 1 

Ingress time (hours): 1.0000 

Egress time (hours): 1.0000 

Scheduled on station time (hours): 5.0000 

Platform turn time (hours): 0.5000 

Time to complete scheduled maintenance (hours): 7.0000 

Mean repair time for each non-mission affecting failures (hours): 0.5000 
Flight time between scheduled maintenance actions (hours): 50.0000 
Mean time between non-mission affecting failures (hours): 5.0000 
The z value used to calculate confidence intervals: 1.9600 


DISTRIBUTIONS and Parameters: 

Platform MAF Time: Weibull : 

Platform MAF Time Parameters: 0.75 16.7977 

Sensor Package MAF Time: Exponential 

Sensor Package MAF Time Parameters: 110.0 

Platform Repair Time: LogNormal 

Platform Repair Time Parameters: 0.6616225 0.2510964 
Sensor Package Repair Time: LogNormal 

Sensor Package Repair Time Parameters: 0.6616255 0.2510964 
Logistics Delay Time: LogNormal 

Logistics Delay Time Parameters: -0.7246719 0.2510964 
Preventive Maintenance Time: LogNormal 

Preventive Maintenance Time Parameters: -0.6983266 0.107785 


Table 24: Input Data for Endurance Comparison 
Figure 24 shows the results of this comparison. The return for increasing the number 
of hours of endurance decreases substantially after the increase from six to seven hours. As 
can be seen, there is very little return after six hours. Thus, for an OPTEMPO requirement of 
12 hours, a baseline of 4 UAVs each with an endurance of 6 hours appears to be a sound 


alternative. 
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OPTEMPO Achievement and Endurance 


Time to Achieve Optempo (hours) 





3 4 5 6 7 8 9 
Endurance (hrs) 


Figure 24: Benefit of UAV Endurance 


K. ANALYSIS OF ALTERNATIVES. 

For the next example, we assume that an acquisition decision must be made about 
three UAV systems. Furthermore, developers have data that can be input into UAVSim. 
Analysts can use this model to evaluate the systems and compare their performance. For the 
purposes of this example, three hypothetical systems are to be evaluated, and each system has 
different capabilities. Table 25 shows the major differences in the systems. One of the 


systems examined had characteristics equivalent to the objective values listed in the ORD. 
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Table 25: Characteristics of Systems 





In the runs of the simulation, the ingress time was held constant for each of the 
systems so that the mission requirement would be identical. The question that must be 
answered is: "Which UAV system performs best?" An example of one of the input files is 
shown in Table 26. Table 27 gives the mean ETOS and 95% confidence interval and Figure 


25 gives a graphical depiction of the ETOS obtained with each system. 


T/ 





SIMULATION Input data for \code\testi5\altl.txt.out: 

Number of platforms: 4 

Number of maintenance paths: 2 

Priority maintenance active: true 

Wear and tear allowed on UAVs: true 

Preventive Maintenance Effectiveness: 0.0 

Maintenance constraints active: false 

Number of maintenance teams: N/A 

Maximum number of platforms in flight :2 

GCS constraints active: false 

Number of GCS crews: 1 

Length of Warm Up Period (days): 360.0000 

Planned Length of Deployment (days): 30.0000 

Actual Length of Simulation (days): 390.0000 

OPTEMPO constraints active: false 

Deployment OPTEMPO (hours): 12.0000 

Surge OPTEMPO constraints active: false 

Surge OPTEMPO (hours): 8.0000 

Ingress time (hours): 1.0000 

Egress time (hours): 1.0000 

Scheduled on station time (hours): 6.0000 

Platform turn time (hours): 0.5000 

Time to complete scheduled maintenance (hours): 7.0000 

Mean repair time for each non-mission affecting failures (hours): 0.5000 
Flight time between scheduled maintenance actions (hours): 50.0000 
Mean time between non-mission affecting failures (hours): 5.0000 
The z value used to calculate confidence intervals: 1.9600 


DISTRIBUTIONS and Parameters: 

Platform MAF Time: Weibull 

Platform MAF Time Parameters: 0.75 50.393093 

Sensor Package MAF Time: Exponential 

Sensor Package MAF Time Parameters: 110.0 

Platform Repair Time: LogNormal 

Platform Repair Time Parameters: 0.6616225 0.2510964 
Sensor Package Repair Time: LogNormal 

Sensor Package Repair Time Parameters: 0.6616255 0.2510964 
Logistics Delay Time: LogNormal 

Logistics Delay Time Parameters: -0.7246719 0.2510964 
Preventive Maintenance Time: LogNormal 

Preventive Maintenance Time Parameters: -0.6983266 0.107785 


Table 26: Input Data for Analysis of Alternatives 
Maintenance constraints were not active. Therefore, each maintenance path had its own 
maintenance team with no limitations. 
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System Mean ETOS 
ETOS 95% Confidence Interval 


UAV - ORD 0.9236 0.9080 - 0.9391 
UAV -B 0.9903 0.9869 - 0.9939 
UAV-C 0.9844 0.9759 - 0.9928 


Table 27: Result of Analysis of Alternatives 









By inspection, the confidence intervals for UAV B and UAV C overlap and suggest that there 


is no significant difference between the two systems’ mean ETOS. 


Comparison of UAV Systems 





UAV - ORD UAV -B UAV-C 
UAV System 


Figure 25: Comparison of Alternatives 


Based on the values of ETOS, it appears that all of the systems perform relatively 
well. However, UAV B has the best performance. While UAVSim can be used as analysis 


tool to assist decision-makers it should not be the sole tool. For example, although UAV - B 
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had the best performance, a cost analysis of UAV B and UAV C or other selection criteria 
may result in C being selected as the best system. 


L. SUMMARY. 


This chapter has demonstrated several uses of an expandable, stochastic, discrete- 
event simulation, UAVSim, to support SBA and doctrinal analysis. UAVSim can be used to 
determine values of parameters to enhance performance, to determine if enhancement is 
possible, and also to determine points of diminishing return. Our analysis also indicated that 
specification of only a mean value for MTBMAF in the ORD might not be sufficient for 
reliable performance. The flexibility and reusability of this simulation have also been shown. 
The analyst can choose from a variety of distributions to use for all of the stochastic variables 
within the model or add his own. This model can also be used for analysis of alternatives, 


offering substantial benefit to the STEP of SBA as it applies to UAVs. 
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V. RECOMMENDATIONS. 





A. GENERAL. 

The model developed in this thesis is designed to serve as a basis for additional 
analysis involving the TUAV and other aerial platforms. The comparisons shown provide 
examples of how this model could be used to answer specific questions during the acquisition 
process, provide indications of performance during operational missions, and the effectiveness 
of changes in system structure. This model has been designed for use during all phases of the 
acquisition process and is flexible enough for the analyst to perform a variety of "what if" 
analyses. Perhaps one of the most substantial benefits of a model such as UAVSim is that it is 
coded in Java which allows a variety of extensible applications. This simulation is by no 
means all encompassing and can be improved upon. This chapter discusses general areas 
within the model that warrant further development and provides topics for further study. 

B. - MODEL IMPROVEMENT. 

Because this model was designed to serve as a basis for further analysis, the list of 
possible improvements is endless. The topics listed below are some of the more important 
possible improvements. 

1. Various Aerial Platforms. 

This study has focused on the use of UAVSim for the analysis of TUAVs; however, it 
can be used in the analysis of other UAVs as well. The structure of this simulation is such that 
it can be easily expanded for use with virtually all types of UAVs. Moreover, it can be 


expanded for analysis of manned aircraft as well, such as the Commanche Helicopter. 
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2. Four Dimensions. 

The flight of UAVs within the current version of the model is based only on time. At 
any specific instance, an exact location of an entity cannot be determined by geographical 
reference. The addition of geographic coordinates along with time will allow more robust 


analysis. This type of enhancement will further facilitate the realism associated with a combat — 


scenario. 

3. Area Search and Detection. 

This simulation does not model area search or the detection of targets. The search and 
detection process is detailed and complex. An implicit detection methodology could be used; 
however, this model would benefit from the explicit representation of search and ieiiglies in 
the evaluation of the performance of UAVs once a geo-reference is incorporated. ETOS is not 
a measure of the effectiveness of a UAV's search and detection capability. 

C. TOPICS OF FURTHER STUDY. 

While conducting research for and performing this study, several topics of interest 
were identified. The following list identifies several of these topics. 

1. Effectiveness of UAV Sensors. 

Perhaps the ultimate question that a commander would like the answer to when it 
comes to a new system is "How effective is this system?" The work that has been done in this 
study can be used as a basis for further study in the effectiveness of UAVs. A proposed 
methodology would be to build a combat scenario and link an expanded version of UAVSim 
to it. The expanded version of UAVSim should be capable of the detection of targets and 


sending that information to firing units. 
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There is a two-fold benefit for such a study. One, it would provide an approximate 
indication of the effectiveness of a UAV system in a particular combat scenario. Secondly, it 
would provide a means for determining sensor-shooter timelines. 

2. Sensor-Shooter Timeline. — 

Often the difference in hitting and missing a target on the battlefield is the timeliness 
in which the target is prosecuted. Such a study would provide an approximate indication of 
the timeliness in which a target must be engaged. Furthermore, it might suggest changes in 
training and TTP. In the training arena, such a study may indicate the need for greater 
proficiency for operators in passing information from sensor to shooter. Still yet, it may 
suggest changes in the manner in which targets are engaged. 

3. Dynamic Retasking. 

While conducting research at the 4th Infantry Division, FT Hood, Texas, I found that 
one of the recurring topics in the use of UAVs was dynamic retasking. Dynamic retasking is 
the unplanned diversion of a UAV to perform another mission. Several members of the 
division staff were interested in the effect of dynamic retasking given a limited number of 


UAVs which have limited hours of endurance. 
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VI. SUMMARY 

This thesis has demonstrated the development and use of an expandable, stochastic 
discrete-event simulation, UAVSim. This model was developed on a personal computer using 
Java and the discrete-event library Simkit. It has been shown that UAVSim can be used to 
determine values of parameters to enhance performance, determine if enhancement is 
possible, and also determine the point of diminishing return. Analysis also indicated that 
specification of only a mean value for MTBMAF in the ORD might not be sufficient for 
reliable performance. In addition, this model can be used for analysis of alternatives. This 
feature offers substantial benefit to SBA as it applies to UAVs. Lastly, UAVSim is scalable, 
expandable and reusable. It can be used throughout all phases of the acquisition process and 
beyond. 

This thesis serves as a basis for follow-on studies involving the TUAV and other 
UAVs. Recommendations for model improvement and examples of follow-on studies have 


been provided. 
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APPENDIX A. EXAMPLE MODEL OUTPUT FILES 
Table A-1 shows an example of an input file which analysts can use to verify the 
inputs of the model. Table A-2 shows the output statistics file. A mean and user specified 


confidence interval is presented for each MOP. 





SIMULATION Input data for \code\test1\lie20maf.txt.out: 
Number of platforms: 4 

Number of maintenance paths: 1 

Priority maintenance active: false 

Wear and tear allowed on UAVs: false 

Preventive Maintenance Effectiveness: 0.0 

Maintenance constraints active: false 

Number of maintenance teams: 1 

Maximum number of platforms in flight :2 

GCS constraints active: false 

Number of GCS crews: 1 

Length of Warm Up Period (days): 230.0000 

Planned Length of Deployment (days): 5.0000 

Actual Length of Simulation (days): 235.0000 

OPTEMPO constraints active: false 

Deployment OPTEMPO (hours): 12.0000 

Surge OPTEMPO constraints active: false 

Surge OPTEMPO (hours): 8.0000 

Ingress time (hours): 1.0000 

Egress time (hours): 1.0000 

Scheduled on station time (hours): 4.0000 

Platform turn time (hours): 0.5000 

Time to complete scheduled maintenance (hours): 7.0000 
Mean repair time for each non-mission affecting failures (hours): 0.5000 
Flight time between scheduled maintenance actions (hours): 50.0000 

Mean time between non-mission affecting failures (hours): 5.0000 

The z value used to calculate confidence intervals: 1.9600 



































DISTRIBUTIONS and Parameters: 
Platform MAF Time: Exponential 

Platform MAF Time Parameters: 20.0 

Sensor Package MAF Time: Exponential 

Sensor Package MAF Time Parameters: 110.0 
Platform Repair Time: Exponential 

Platform Repair Time Parameters: 2.0 

Sensor Package Repair Time: Exponential 
Sensor Package Repair Time Parameters: 2.0 
Logistics Delay Time: Exponential 

Logistics Delay Time Parameters: 0.5 
Preventive Maintenance Time: Exponential 
Preventive Maintenance Time Parameters: 0.5 














Table A-1: Example Input Data 
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\code\test1\1lie20maf.txt.stats 
UAV Company Statistics 


DEPLOYMENT Statistics: 

Number of Days per Deployment: 235.0 

Mean ETOS: 0.9048 

Standard Deviation of ETOS (hours): 0.0069 
Confidence Interval: 0.8913 - 0.9182 


OPTEMPO Statistics: 
OPTEMPO not used; not OPTEMPO Statistics 


SORTIE Statistics: 

Mean Number of Sorties per Deployment: 1481.4000 

Standard Deviation for Sorties per Deployment: 10.1152 

Confidence Interval for Sorties Per Deployment: 1461.5742 - 1501.2258 


Mean Sortie Generation Rate (sorties/day): 6.3038 
Standard Deviation for Sortie Generation Rate (sorties/day): 0.0430 
Confidence Interval for Sorties Generation Rate: 6.2195 - 6.3882 


MAINTENANCE Statistics: 

Prioritization of Maintenance Active: false 

Mean Wait Time in Maintenance Queue (hours): 1.7914 

Standard Deviation of Mean Wait Time in Queue (hours): 0.0868 

Confidence Interval for Mean Wait Time in Queve (hours): 1.6214 - 1.9615 


Mean Time UAVs are Unavailable (hours): 3.2700 
Standard Deviation of Mean Time UAVs are Unavailable (hours): 0.2230 
Confidence Interval for Mean Time UAVs are Unavailable (hours): 2.8329 - 3.7071 


Mean Amount Time UAVs Fly without Failures (hours): 5.1812 
Standard Deviation for Amount of Time UAVs Fly without Failures (hours): 0.0394 
Confidence Interval for Amount of Time UAVs Fly without Failures (hours): 5.1041 - 5.2584 


Mean Time UAVs are Down for Maintenance (hours): 2.6408 

Standard Deviation of Maintenance Down Time (hours): 0.0298 

Confidence Interval for Maintenance Down Time (hours): 2.5825 - 2.6991 
Mean Percentage of Scheduled Services: 0.1066 


Standard Deviation of Mean Percentage of Scheduled Services: 0.0008 
Confidence Interval for Mean Percentage of Scheduled Services: 0.1051 - 0.1082 


Table A-2: Example Statistics Output File 
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APPENDIX B. VERIFICATION OF MOP NORMALITY ASSUMPTION 
Figure B-1 shows a plot of the empirical and hypothesized CDFs for ETOS. The 


empirical CDF is represented by the blocked line and the hypothesized CDF is shown as the 


more smooth line. 


Comparison of Empirical and Hypothesized CDFs 


probability 





0.325 0.330 0.335 0.340 0.345 
blocked line is the emipirical distribution 


Figure B-1: Exploratory Plot of Normal CDFs 
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APPENDIX C. GENERATION OF DISTRIBUTIONS 


To allow for greater flexibility in selection of the distributions in UAVSim, Weibull 





and Lognormal distributions were constructed. 

Weibull Distribution. 

The distribution for Weibull variates was constructed using the inverse-transform 
algorithm. The algorithm for the Weibull was obtained from Simulation Modeling and | 
Analysis [Ref. 26:p. 490]. 

Define U ~ uniform(0,1) as a random variate which is iid. The uniform(0,1) random variate 
was generated using a previously existing class in Simkit. The author assumed that the 
implementation of the algorithm to generate uniform(0,1) iid random variates to be correct. 

1. generate U 

2. return X, where X = B(-InU)'”” 

The resulting values for X are Weibull variates and the observations are iid 

Lognormal Distribution 

The algorithm for the Lognormal was obtained from Simulation Modeling and 
Analysis [Ref. 26:p. 492]. 


Define Y~normal(j1,c) random variate which is iid. The author assumes that the 
normal(1,o) distribution is implemented correctly. 


1. generate Y 


2. retum X=e* 
The resulting values for X are iid Lognormal random variates. _ 
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APPENDIX D: EXAMPLES OF WEIBULL DISTRIBUTIONS 


Figure D-1 shows the Weibull distributions that were used in this study. This example 


only illustrates distributions with a mean of twenty. 


Weibull Distribution, E[X] = 20, alpha= 0.25 Weibull Distribution, E[X] = 20, alpha= 0.75 
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Figure D-1: Weibull Distributions 


The proportion shown in each of the plots is the probability of surviving past the 


MTBMAF. This highlights the importance of not merely specifying the mean in the ORD. 
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APPENDIX E: SCULPTURED AND TRIANGULAR DISTRIBUTIONS 
Sculptured(a). 
The algorithm for the Sculptured(a) distribution is: 


Define U ~ uniform(0,1) random variate which is iid. The Uniform(0,1) random variate was 
generated using Simkit's random number generator. 


1. generate U 

2. return X, where X = -InU*(1 + (a * -InU)) 

3: 

The resulting values for X are Sculptured(a) variates and the observations are iid [Ref 22:p. 
35]. 

Sculptured(b). 

The algorithm for the Sculptured(b) distribution is: 


Define U ~ uniform(0,1) random variate which is iid. 


1. generate U 
2. return X, where X = -InU*(1 + (a * -(nU)’)) 


The values for X are Sculptured(b) variates and the observations are iid [Ref 22:p. 35]. 
Triangular(a, b, c). 

The algorithm for the Triangular distribution was verified as part of course work done at the 
Naval Postgraduate School. Diagnostic plots and a test for autocorrelation were done to verify 


that this algorithm worked properly as long asa<c <b. 


define U1 ~ uniform(0,1) random variate which is iid. 
define U2 ~ uniform(0,1) random variate which is iid. 


1. generate U1 and U2 
2. if (U1 <1- (c-a)/(b-a)) { 
X =b - sqrt((a-b)* - U2(b-a)’) 


else { 
X =a + sqrt(U2(b-a)) 
} 
3. return X 


The resulting variates are Triangular (a, b, c) and are iid. 
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